Intra prediction encoding/decoding method and apparatus for chrominance components

ABSTRACT

Disclosed is an image decoding method using the correlation between color components to perform into prediction of chrominance components. Here, the image decoding method using the correlation between color components to perform intra prediction of chrominance components comprises the steps of: checking image data and a prediction mode in a bitstream; generating a prediction block according to a reconstructed prediction mode; determining compensation settings according to the size of a current block and the reconstructed prediction mode; compensating the prediction block according to the determined compensation settings; and reconstructing the current block by adding reconstructed image data to the prediction block.

RELATED APPLICATIONS

This application is a divisional application of U.S. patent applicationSer. No. 17/109,135, filed Dec. 1, 2020, which is a divisionalapplication of U.S. patent application Ser. No. 16/880,788, filed May21, 2020, which is a continuation application of the internationalapplication No. PCT/KR2019/000436, filed Jan. 11, 2019, which claimspriority to the Korean patent application No. 10-2018-0005294, filedJan. 15, 2018. All of these applications are incorporated by referenceherein in their entireties.

TECHNICAL FIELD

The present invention relates to an image encoding/decoding method andapparatus for chrominance components. More specifically, it relates to amethod and apparatus for generating prediction blocks based oncorrelation information between color components, and reducingdeterioration between blocks by applying correction to the generatedprediction blocks.

BACKGROUND

With the spread of the Internet and portable terminals and thedevelopment of information communication technology, the use ofmultimedia data is rapidly increasing. Accordingly, the need forimproving the performance and efficiency of an image processing systemhas been significantly increased to perform various services or tasksthrough image prediction in various systems, but research anddevelopment results that can respond to this atmosphere areinsufficient.

As described above, in the conventional image encoding and decodingmethod and apparatus, performance improvement in image processing,particularly image encoding or image decoding, is required.

SUMMARY

An object of the present invention for solving the above problems is toprovide an image encoding/decoding method and apparatus for performingintra prediction by utilizing a correlation between color components.

A method of decoding an image according to an embodiment of the presentinvention for achieving the above object may comprise checking imagedata and a prediction mode in a bitstream, generating a prediction blockaccording to a restored prediction mode, determining a correctionsetting according to a size of a current block and the restoredprediction mode, compensating the prediction block according to thedetermined correction settings, and restoring the current block byadding the reconstructed image data and the prediction block.

Herein, the step of determining the correction setting may furthercomprise determining whether to perform the correction according to thesize of the current block and a type of the prediction mode.

Herein, the step of determining the correction setting may furthercomprise determining a region to be corrected according to the size ofthe current block and the type of prediction mode.

When using a method for performing intra prediction by utilizing acorrelation between color components according to the present inventionas described above, prediction accuracy is high and encoding performancecan be improved.

In addition, since correction is performed on a boundary region of aprediction block, there is an advantage that block degradation can bereduced.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a conceptual diagram of an image encoding and decoding systemaccording to an embodiment of the present invention.

FIG. 2 is a block diagram of an image encoding apparatus according to anembodiment of the present invention.

FIG. 3 is a block diagram of an image decoding apparatus according to anembodiment of the present invention.

FIG. 4 is an exemplary diagram illustrating an intra prediction modeaccording to an embodiment of the present invention.

FIG. 5 is a conceptual diagram illustrating intra prediction for adirectional mode and a non-directional mode according to an embodimentof the present invention.

FIG. 6 is a conceptual diagram illustrating intra prediction regarding acolor copy mode according to an embodiment of the present invention.

FIG. 7 is an exemplary diagram illustrating a corresponding block ofeach color space and a region adjacent thereto in relation to a colorcopy mode according to an embodiment of the present invention.

FIG. 8 is an exemplary diagram for explaining a reference pixelconfiguration used for intra prediction according to an embodiment ofthe present invention.

FIG. 9 is a conceptual diagram illustrating a block adjacent to a targetblock for intra prediction according to an embodiment of the presentinvention.

FIG. 10 is a flowchart for explaining an implementation example of animage encoding method according to an embodiment of the presentinvention.

FIG. 11 is a flowchart for explaining an example of an implementation ofan image decoding method according to an embodiment of the presentinvention.

FIG. 12 is an exemplary diagram for detailed setting of a color copymode according to an embodiment of the present invention.

FIG. 13 is a conceptual diagram illustrating various configurationexamples of pixels to be corrected in a current block according to anembodiment of the present invention.

FIG. 14 is an exemplary diagram for explaining a case where correctionis performed in a color copy mode according to an embodiment of thepresent invention.

FIG. 15 is an exemplary diagram for explaining a case where correctionis performed in a color copy mode according to an embodiment of thepresent invention.

DETAILED DESCRIPTION

A method of decoding an image according to an embodiment of the presentinvention for achieving the above object may comprise checking imagedata and a prediction mode in a bitstream, generating a prediction blockaccording to a restored prediction mode, determining a correctionsetting according to a size of a current block and the restoredprediction mode, compensating the prediction block according to thedetermined correction settings, and restoring the current block byadding the reconstructed image data and the prediction block.

Herein, the step of determining the correction setting may furthercomprise determining whether to perform the correction according to thesize of the current block and a type of the prediction mode.

Herein, the step of determining the correction setting may furthercomprise determining a region to be corrected according to the size ofthe current block and the type of prediction mode.

The present invention can be applied to various changes and can havevarious embodiments, and specific embodiments will be illustrated in thedrawings and described in detail. However, this is not intended to limitthe present invention to specific embodiments, and should be understoodto include all modifications, equivalents, and substitutes included inthe idea and technology scope of the present invention.

Terms such as first, second, A, and B may be used to describe variouscomponents, but the components should not be limited by the terms. Theterms are used only for the purpose of distinguishing one component fromother components. For example, a first component may be referred to as asecond component without departing from the scope of the presentinvention, and similarly, the second component may be referred to as thefirst component. The term and/or includes a combination of a pluralityof related described items or any one of a plurality of relateddescribed items.

When an element is said to be “linked” or “connected” to anotherelement, it may be directly linked or connected to other components, butit should be understood that other components may exist in the middle.On the other hand, when a component is said to be “directly linked” or“directly connected” to another component, it should be understood thatno other component exists in the middle.

The terms used in the present invention are only used to describespecific embodiments, and are not intended to limit the presentinvention. Singular expressions include plural expressions unless thecontext clearly indicates otherwise. In the present invention, termssuch as “include” or “have” are intended to indicate that there arefeatures, numbers, steps, operations, components, parts, or combinationsthereof described in the specification, and it should be understood thatone or more other features or numbers, steps, actions, components,parts, or combinations thereof are not excluded in advance.

Unless otherwise defined, all terms used herein, including technical orscientific terms, mean the same as generally understood by a personskilled in the art to which the present invention pertains. Terms, suchas those defined in a commonly used dictionary, should be interpreted asbeing consistent with meanings in the context of related technologies,and are not to be interpreted as ideal or excessively formal meaningsunless explicitly defined in the present invention.

Typically, one or more color spaces may be configured according to acolor format of an image. It may be composed of one or more pictureshaving a certain size or one or more pictures having a different sizeaccording to a color format. For example, color formats such as 4:4:4,4:2:2, 4:2:0, and Monochrome (consisting only of Y) may be supported inthe YCbCr color configuration. For example, in the case of YCbCr 4:2:0,it may be composed of one luminance component (Y in this example, Y) andtwo chrominance components (Cb/Cr in this example). Herein, thecomposition ratio of the chrominance component and the luminancecomponent may have a horizontal and vertical ratio of 1:2. For example,in the case of 4:4:4, it may have the same aspect ratio horizontally andvertically. When configured as one or more color spaces as in the aboveexample, the picture may be divided into each color space.

Images can be classified into I, P, B, etc. according to the image type(e.g., picture type, slice type, tile type, etc.). Herein, the I imagetype may mean an image that is self-decoded/decoded without using areference picture, the P image type may mean an image that isencoded/decoded using a reference picture but only allows forwardprediction, and the B image type may mean an image that allowsforward/backward prediction by performing encoding/decoding using areference picture. In addition, depending on encoding/decoding settings,some of the types may be combined (combining P and B) or image types ofdifferent configurations may be supported.

FIG. 1 is a conceptual diagram of an image encoding and decoding systemaccording to an embodiment of the present invention.

Referring to FIG. 1, the image encoding apparatus 105 and the decodingapparatus 100 may be a Personal computer (PC), a Notebook Computer, aPersonal Digital Assistant (PDA), a Portable Multimedia Player (PMP), aPlayStation Portable (PSP), a Wireless Communication Terminal, a userterminal such as a smart phone or a TV, or a server terminal such as anapplication server and a service server, and may include a variety ofdevices having communication devices such as communication modems forcommunication with various devices or wired and wireless communication,memory (120, 125) for storing various programs and data for inter orintra prediction for encoding or decoding an image, a processor (110,115) for calculating and controlling through executing a program, or thelike.

In addition, an image encoded as a bitstream by the image encodingapparatus 105 may be transmitted to the image decoding apparatus 100 inreal-time or non-real-time through the Internet, short-range wirelesscommunication network, wireless LAN network, WiBro network or mobilecommunication network, or through various communication interfaces suchas cable or Universal Serial Bus (USB), and may be decoded,reconstructed as an image, and reproduced in the image decodingapparatus 100. In addition, an image encoded in a bitstream by the imageencoding apparatus 105 may be transmitted from the image encodingapparatus 105 to the image decoding apparatus 100 through acomputer-readable recording medium.

The above-described image encoding device and image decoding device maybe separate devices, but may be made into one image encoding/decodingdevice depending on implementation. In that case, some components of theimage encoding apparatus may be implemented to include at least the samestructure or perform at least the same functions as substantially thesame technical elements as some components of the image decodingapparatus.

Therefore, in the detailed description of the following technicalelements and their operating principle, duplicate description ofcorresponding technical elements will be omitted. In addition, since theimage decoding apparatus corresponds to a computing apparatus thatapplies an image encoding method performed by the image encodingapparatus to decoding, the following description will focus on the imageencoding apparatus.

The computing device may include a memory that stores a program orsoftware module that implements an image encoding method and/or an imagedecoding method, and a processor that is connected to the memory andperforms a program. Herein, the image encoding apparatus may be referredto as an encoder, and the image decoding apparatus may be referred to asa decoder, respectively.

FIG. 2 is a block diagram of an image encoding apparatus according to anembodiment of the present invention.

Referring to FIG. 2, the image encoding apparatus 20 may include aprediction unit 200, a subtraction unit 205, a transformation unit 210,a quantization unit 215, an inverse quantization unit 220, and aninverse transformation unit 225, an adder 230, a filter unit 235, anencoded picture buffer 240, and an entropy encoder 245.

The prediction unit 200 may be implemented using a prediction module,which is a software module, and may generate a prediction block by usingan intra prediction method or an inter prediction method for blocks tobe encoded. The prediction unit 200 may generate a prediction block bypredicting a current block to be currently encoded in the image. Inother words, the prediction unit 200 may generate a prediction blockhaving a prediction pixel value (predicted pixel value) of each pixelgenerated by predicting a pixel value of each pixel of the current blockto be encoded in an image according to intra or inter prediction. Inaddition, the prediction unit 200 may transmit information necessary forgenerating a prediction block, such as information about a predictionmode, such as an intra prediction mode or an inter prediction mode, tothe encoding unit, to cause the encoding unit to encode informationabout the prediction mode. Herein, a processing unit for whichprediction is performed, and a processing unit for which the predictionmethod and specific contents are determined may be determined accordingto encoding/decoding settings. For example, a prediction method, aprediction mode, and the like are determined in a prediction unit, andprediction may be performed in a transformation unit.

In the inter prediction unit, a translation motion model and anon-translation motion model may be divided according to a motionprediction method. In the case of the translation motion model,prediction can be performed considering only parallel movement, and inthe case of a non-translation movement model, prediction can beperformed considering movement such as rotation, perspective, and zoomin/out as well as parallel movement. Assuming unidirectional prediction,one motion vector may be required for the translation motion model, butone or more motion vectors may be required for the non-translationmotion model. In the case of the non-translation motion model, eachmotion vector may be information applied to a preset position of thecurrent block, such as an top left vertex and a top right vertex of thecurrent block, and the position of a region to be predicted of thecurrent block through the corresponding motion vector may be acquired inunits of pixels or sub-blocks. In the inter prediction unit, someprocesses described below may be applied in common and some otherprocesses may be individually applied according to the motion model.

The inter prediction unit may include a reference picture constructionunit, a motion estimation unit, a motion compensation unit, a motioninformation determination unit, and a motion information encoding unit.The reference picture construction unit may include pictures encodedbefore or after the current picture in reference picture lists L0 andL1. A prediction block may be obtained from the reference pictureincluded in the reference picture list, and a current picture may alsobe configured as a reference picture according to an encoding settingand included in at least one of the reference picture lists.

In the inter prediction unit, the reference picture construction unitmay include a reference picture interpolation unit, and may perform aninterpolation process for a decimal pixel unit according tointerpolation precision. For example, an 8-tap DCT-based interpolationfilter may be applied to a luminance component, and a 4-tap DCT-basedinterpolation filter may be applied to a chrominance component.

In the inter prediction unit, the motion estimation unit may be aprocess of searching for a block having a high correlation with acurrent block through a reference picture, and various methods such asfull search-based block matching algorithm (FBMA) and three step search(TSS) may be used. In addition, the motion compensation unit means aprocess of obtaining a prediction block through a motion estimationprocess.

In the inter prediction unit, a motion information determination unitmay perform a process for selecting optimal motion information of acurrent block, and the motion information may be encoded by a motioninformation encoding mode such as Skip Mode, Merge Mode, and CompetitionMode. The mode may be configured by combining a supported mode accordingto a motion model, and a skip mode (translation), a skip mode (otherthan translation), a merge mode (translation), a merge mode (other thantranslation), a competition mode (translation), and a competition mode(other than translation) can be an example for it. Depending on anencoding setting, some of the modes may be included in a candidategroup.

A motion information encoding mode may obtain a motion informationprediction value (motion vector, reference picture, predictiondirection, etc.) of a current block from at least one candidate block,and when two or more candidate blocks are supported, optimal candidateselection information can occur. In the skip mode (no residual signal)and the merge mode (there is a residual signal), a prediction value maybe used as motion information of the current block, and in thecompetition mode, difference information between the motion informationof the current block and the prediction value may occur.

A candidate group for a motion information prediction value of a currentblock may be constructed adaptively and variously according to a motioninformation encoding mode. Motion information of a block (for example, aleft, top, top left, top right, bottom left block, etc.) spatiallyadjacent to the current block may be included in the candidate group,and motion information of a block temporally adjacent to the currentblock may be included in the candidate group, and mixed motioninformation of a spatial candidate and a temporal candidate may beincluded in the candidate group.

The temporally adjacent block may include a block in another imagecorresponding to the current block, and may mean a block located in aleft, right, top, bottom, top left, top right, bottom left, bottom rightblock, or the like. of the block. The mixed motion information may meaninformation obtained as an average, a median, etc. through motioninformation of spatially adjacent blocks and motion information oftemporally adjacent blocks.

There may be a priority order for constructing a candidate group of amotion information prediction value. The order included in aconfiguration of the candidate group of the prediction value may bedetermined according to the priority order, and the configuration of thecandidate group may be completed when the number of candidate groups(determined according to the motion information encoding mode) is filledaccording to the priority order. Herein, the priority order may bedetermined in the order of motion information of spatially adjacentblocks, motion information of temporally adjacent blocks, and mixedmotion information of spatial candidates and temporal candidates, butother modifications are also possible.

For example, among spatially adjacent blocks, it may be included in acandidate group in the order of left-top-top right-bottom left-top leftblock, etc., and among the temporally adjacent blocks, it may beincluded in a candidate group in the order of bottomright-middle-right-bottom block, etc.

The subtraction unit 205 may generate a residual block by subtracting aprediction block from a current block. In other words, the subtractionunit 205 may generate a residual block, which is a residual signal inthe form of a block, by calculating a difference between a pixel valueof each pixel of the current block to be encoded and a prediction pixelvalue of each pixel of the prediction block generated through theprediction unit. In addition, the subtraction unit 205 may generate theresidual block according to a unit other than a block unit obtainedthrough the block division unit described later.

The transformation unit 210 may convert a signal belonging to a spatialdomain into a signal belonging to a frequency domain, and the signalobtained through a transform process is called a transformedcoefficient. For example, a residual block having a residual signalreceived from the subtraction unit may be transformed to obtain atransform block having a transformed coefficient, and an input signal isdetermined according to encoding settings, which is not limited to theresidual signal.

The transformation unit can transform the residual block using transformtechniques such as Hadamard Transform, Discrete Sine Transform (DSTBased-Transform), and Discrete Cosine Transform (DCT Based-Transform).However, the present invention may not be limited thereto, and variousconversion techniques that improve and modify it may be used.

At least one of the transformation techniques may be supported, and atleast one detailed transformation technique may be supported in eachtransformation technique. In this case, the detailed transformationtechnique may be a transformation technique in which some of basevectors are configured differently in each transformation technique.

For example, in the case of DCT, one or more detailed transformationtechniques of DCT-I to DCT-VIII may be supported, and in the case ofDST, one or more detailed transformation techniques of DST-I to DST-VIIImay be supported. Some of the detailed transformation techniques may beconfigured to configure a candidate group for a transformationtechnique. For example, DCT-II, DCT-VIII, and DST-VII may be configuredas the candidate group of the transformation technique to performtransformation.

The transformation can be performed in the horizontal/verticaldirection. For example, a pixel value in a spatial domain can beconverted into a frequency domain by performing a total two-dimensionaltransformation which is performing a one-dimensional transformation inthe horizontal direction using the transformation technique of DCT-IIand a one-dimensional transformation in the vertical direction using thetransformation technique of DST-VIII.

Transformation can be performed using one fixed transformationtechnique, or transformation can be performed by adaptively selecting atransformation technique according to encoding/decoding settings.Herein, in the adaptive case, a transform technique may be selectedusing an explicit or implicit method. In the explicit case, eachtransformation technique selection information or transformationtechnique set selection information applied to the horizontal andvertical directions may occur in a unit such as a block. In the implicitcase, an encoding setting may be defined according to an image type(I/P/B), color component, block size, shape, and intra prediction mode,and a predefined transformation technique may be selected accordingly.

In addition, it may be possible that some of the transformations areomitted depending on encoding settings. This means that one or more ofthe horizontal/vertical units can be omitted, either explicitly orimplicitly.

In addition, the transformation unit may transmit information necessaryfor generating a transform block to the encoding unit to encode it,record the encoded information to a bitstream, and transmit it to adecoder, and a decoding unit of the decoder may parse the transmittedinformation and use it in the process of an inverse transformation.

The quantization unit 215 may quantize an input signal, and a signalobtained through a quantization process is called a quantizedcoefficient. For example, a quantization block having a quantizedcoefficient may be obtained by quantizing a residual block having aresidual transformed coefficient received from the transformation unit,and the input signal is determined according to encoding settings, whichare not limited to the residual transform coefficient.

The quantization unit may quantize a transformed residual block using aquantization technique such as Dead Zone Uniform Threshold Quantization,Quantization Weighted Matrix, etc., but it may not be limited thereto,and various quantization techniques that improve and modify it may beused.

Depending on encoding settings, a quantization process can be omitted.For example, the quantization process (including its inverse process)may be omitted according to encoding settings (e.g., a quantizationparameter is 0. that is, a lossless compression environment). As anotherexample, if compression performance through quantization is not achievedaccording to characteristics of an image, the quantization process maybe omitted. In this case, a region in which the quantization process isomitted among quantization blocks (M×N) may be an entire region or apartial region (M/2×N/2, M×N/2, M/2×N, etc.), and quantization omissionselection information may be determined implicitly or explicitly.

The quantization unit may transmit information necessary for generatinga quantization block to an encoding unit to encode it, record theencoded information to a bitstream, and transmit it to a decoder, and adecoding unit of the decoder may parse the transmitted information anduse it in the process of an inverse quantization.

Although the above example has been described under the assumption thata residual block is transformed and quantized through the transformationunit and the quantization unit, a residual block having transformcoefficients may be generated by transforming a residual signal of theresidual block and a quantization process may not be performed. Inaddition, it is possible not only to perform the quantization processwithout transforming the residual signal into the transform coefficient,but also not to perform both the transformation and the quantizationprocess. This can be determined according to an encoder setting.

The inverse quantization unit 220 inversely quantizes a residual blockquantized by the quantization unit 215. That is, the inversequantization unit 220 inversely quantizes a quantization frequencycoefficient sequence to generate a residual block having a frequencycoefficient.

The inverse transformation unit 225 inversely transforms a residualblock quantized by the inverse quantization unit 220. That is, theinverse transformation unit 225 inversely transforms the frequencycoefficients of the inverse quantized residual block to generate aresidual block having a pixel value, that is, a reconstructed residualblock. Herein, the inverse transformation unit 225 may perform aninverse transform using the transformation method used by thetransformation unit 210 in reverse.

The adder 230 restores a current block by adding the prediction blockpredicted by the prediction unit 200 and the residual block restored bythe inverse transformation unit 225. The reconstructed current block isstored as a reference picture (or reference block) in the encodedpicture buffer 240 and may be used as a reference picture when encodingthe next block, another block, or another picture in the future.

The filter unit 235 may include one or more post-processing filterprocesses such as a deblocking filter, sample adaptive offset (SAO), andadaptive loop filter (ALF). The deblocking filter can remove blockdistortion caused by a boundary between blocks in a reconstructedpicture. The ALF may perform filtering based on a value obtained bycomparing a reconstructed image after a block is filtered through adeblocking filter with an original image. The SAO may restore an offsetdifference from an original image in a unit of a pixel for a residualblock to which a deblocking filter is applied. These post-processingfilters can be applied to the reconstructed picture or block.

The encoded picture buffer 240 may store blocks or picturesreconstructed through the filter unit 235. The reconstructed block orpicture stored in the encoded picture buffer 240 may be provided to theprediction unit 200 that performs intra prediction or inter prediction.

The entropy encoding unit 245 scans the generated quantization frequencycoefficient sequence according to various scanning methods to generate aquantization coefficient sequence, and outputs it by encoding using anentropy encoding technique, and the like. The scan pattern can be set toone of various patterns such as a zigzag, diagonal, and raster. Inaddition, it is possible to generate and output encoded data includingencoding information transmitted from each component in a bitstream.

FIG. 3 is a block diagram of an image decoding apparatus according to anembodiment of the present invention.

Referring to FIG. 3, the image decoding apparatus 30 may be configuredto include an entropy decoding unit 305, a prediction unit 310, aninverse quantization unit 315, an inverse transformation unit 320, anadder/subtractor 325, a filter 330, and a decoded picture buffer 335.

In addition, the prediction unit 310 may include an intra predictionmodule and an inter prediction module again.

First, when an image bitstream transmitted from the image encodingapparatus 20 is received, it may be delivered to the entropy decodingunit 305.

The entropy decoding unit 305 may decode the bitstream and decodedecoding data including quantization coefficients and decodinginformation transmitted to each component.

The prediction unit 310 may generate a prediction block based on datatransmitted from the entropy decoding unit 305. Herein, based on areference image stored in the decoded picture buffer 335, a referencepicture list using a default configuration technique may be constructed.

The inter prediction unit may include a reference picture constructionunit, a motion compensation unit, and a motion information decodingunit, and some may perform the same process as the encoder and some mayperform the inverse process.

The inverse quantization unit 315 may inverse quantize quantizedtransform coefficients provided from a bitstream and encoded by theentropy decoding unit 305.

The inverse transformation unit 320 may generate a residual block byapplying inverse DCT, inverse integer transformation, or similar inversetransformation techniques to transform coefficients.

In this case, the inverse quantization unit 315 and the inversetransformation unit 320 may perform inversely the processes performed bythe transformation unit 210 and quantization unit 215 of the imageencoding apparatus 20 described above, and may be implemented in variousways. For example, the same process and inverse transformation that areshared with the transformation unit 210 and the quantization unit 215may be used, and information about the transformation and quantizationprocess (for example, transformation size and transformation shape,quantization type, etc.) from the image encoding apparatus 20 may beused to perform the transformation and quantization processes inversely.

The residual block that has undergone the inverse quantization andinverse transformation processes may be added to the prediction blockderived by the prediction unit 310 to generate a reconstructed imageblock. The addition may be performed by the adder/subtractor unit 325.

The filter 330 may apply a deblocking filter to remove a blockingphenomenon, if necessary, for a reconstructed image block, andadditional loop filters may also be used before and after the decodingprocess to improve video quality

The reconstructed and filtered image blocks may be stored in the decodedpicture buffer 335.

Although not shown in the drawing, the picture encoding/decoding devicemay further include a picture division unit and a block division unit.

The picture division unit may divide (or partition) a picture into atleast one processing unit such as a color space (e.g., YCbCr, RGB, XYZ,or the like), tile, slice, or basic coding unit (or maximum codingunit), or the like. The block division unit may divide the basic codingunit into at least one processing unit (e.g., coding, prediction,transformation, quantization, entropy, and in-loop filter units).

The basic coding unit may be obtained by dividing pictures in horizontaland vertical directions at regular intervals. Based on this,partitioning of tiles, slices, or the like may be performed, but it maynot be limited thereto. The division unit such as the tile and slice maybe composed of an integer multiple of the basic coding block, but anexception may occur in a division unit located at an image boundary. Forthis, adjustment of the basic coding block size may occur.

For example, a picture may be divided into the division units afterbeing partitioned as a basic coding unit, or a picture may be dividedinto the basic coding units after being partitioned as the divisionunit. In the present invention, it is assumed on the assumption that thepartitioning and division order of each unit is the former, but may notbe limited thereto, and the latter may also be possible depending onencoding/decoding settings. In the latter case, the size of the basiccoding unit may be transformed into an adaptive case according to adivision unit (tile, etc.). That is, it means that a basic coding blockhaving a different size for each division unit can be supported.

In the present invention, a case in which a picture is partitioned intoa basic coding unit is set as a basic setting, and an example describedlater will be described. The default setting may mean that a picture isnot divided into tiles or slices, or a picture is one tile or one slice.However, as described above, when each division unit (tile, slice, etc.)is first partitioned and divided into basic coding units based on theobtained units (i.e., each division unit is not an integer multiple ofthe basic coding unit, etc.), it should be understood that variousembodiments described below may be applied by being the same or changed.

In the case of a slice among the division units, it may be composed of abundle of at least one consecutive block according to a scan pattern,and in the case of a tile, it may be composed of a bundle of spatiallyadjacent blocks in a rectangular shape, and it may be configured by thedefinition of other additional units supported. The slice and the tilemay be a division unit supported for the purpose of parallel processing,etc., and for this, the reference between the division units may belimited (that is, cannot be referred).

In this case, a slice and a tile may be divided into a plurality ofunits according to encoding/decoding settings.

For example, some units <A> may be units including setting informationthat affects the encoding/decoding process (that is, include tileheaders or slice headers), and some units <B> may be units not includingsetting information. Alternatively, some units <A> may be units thatcannot refer to other units in the encoding/decoding process, and someunits <B> may be units that can refer to the other units. In addition,some units <A> may have a vertical relationship including other units<B>, or some units <A> may have an equivalent relationship with otherunits <B>.

Herein, A and B may be a slice and tile (or tiles and slices).Alternatively, A and B may be composed of one of slices and tiles. Forexample, A may be a slice/tile <type 1> and B may be a slice/tile <type2>.

Herein, the type 1 and the type 2 may each be one slice or tile.Alternatively, the type 1 (including the type 2) may be a plurality ofslices or tiles (a set of slices or a set of tiles), and the type 2 maybe a single slice or tile.

As described above, the present invention is described on the assumptionthat a picture is composed of one slice or tile, but when two or moredivision units occur, the above description may be applied to andunderstood in the embodiments described below. In addition, A and B areexamples of characteristics that the division unit may have, andexamples in which A and B of each example are mixed are also possible.

Meanwhile, it may be divided into blocks of different sizes through theblock division unit. Herein, the blocks may be composed of one or moreblocks according to a color format (for example, one luminance block andtwo chrominance blocks, etc.), and the size of the block may bedetermined according to the color format. Hereinafter, for convenienceof description, description will be made based on a block according toone color component (luminance component).

It should be understood that the contents described below are targetedto one color component, but can be applied to other color components inproportion to a ratio (for example, in the case of YCbCr 4:2:0, theratio of the horizontal length to the vertical length of the luminancecomponent and the chrominance component is 2:1) according to the colorformat. In addition, although it is possible to perform block divisiondependent on other color components (for example, depending on a blockdivision result of Y in Cb/Cr), it should be understood that independentblock division may be possible for each color component. In addition,although one common block division setting (considering proportion to alength ratio) can be used, it is necessary to consider and understandthat individual block division settings are used according to colorcomponents.

The block may have a variable size such as M×N (M and N are integerssuch as 4, 8, 16, 32, 64, and 128), and may be a unit (coding block) forperforming encoding. In detail, it may be a unit that is a basis forprediction, transformation, quantization, and entropy encoding, and isgenerally referred to as a block in the present invention. Herein, theblock does not only mean a block of a square, but can be understood as awide concept including various types of regions such as a triangle and acircle, and the present invention will be mainly described in the caseof a square.

The block division unit may be set in relation to each component of animage encoding device and decoding device, and a size and shape of ablock may be determined through this process. Herein, the block to beset may be defined differently according to a component, and aprediction block in a prediction unit, a transformation block in atransformation unit, and a quantization block in a quantization unit maycorrespond to this. However, the present invention may not be limitedthereto, and a block unit according to other components may beadditionally defined. In the present invention, it is assumed on theassumption that the input and output are blocks (i.e., rectangular) ineach component, but in some components, input/output of other shapes(e.g., square, triangle, etc.) may be possible.

A size and shape of the initial (or starting) block of the blockdivision unit may be determined from the higher unit. For example, inthe case of a coding block, the basic coding block may be an initialblock, and in the case of a prediction block, the coding block may be aninitial block. In addition, in the case of a transform block, a codingblock or a prediction block may be an initial block, and this may bedetermined according to encoding/decoding settings.

For example, if an encoding mode is intra, a prediction block may be thehigher unit of the transform block, and if the encoding mode is inter,the prediction block may be a unit independent of the transform block.The initial block, which is the starting unit of division, may bedivided into small-sized blocks, and if the optimal size and shapeaccording to the division of the block are determined, the block may bedetermined as the initial block of the lower unit. The initial block,which is the starting unit of division, can be considered as the initialblock of the higher unit. Herein, the higher unit may be a coding block,and the lower unit may be a prediction block or a transform block, butis not limited thereto. When the initial block of the lower unit isdetermined as in the above example, a dividing process for finding theoptimal size and shape of the block as the higher unit may be performed.

In summary, the block division unit may divide a basic coding unit (orthe maximum coding unit) into at least one coding unit (or lower codingunit). In addition, a coding unit may be divided into at least oneprediction unit, and may be divided into at least one transformationunit. A coding unit may be divided into at least one coding block, thecoding block may be divided into at least one prediction block, anddivided into at least one transform block. A prediction unit may bedivided into at least one prediction block, and a transformation unitmay be divided into at least one transformation block.

In this case, some blocks may be combined with other blocks to performone dividing process. For example, when a coding block and a transformblock are combined as one unit, a dividing process is performed toobtain the optimal block size and shape, which may be not only theoptimal size and shape of the coding block, but also the optimal sizeand shape of the transform block. Alternatively, a coding block and atransform block may be combined in one unit, a prediction block and atransform block may be combined in one unit, and a coding block, aprediction block, and a transform block may be combined in one unit. Inaddition, combinations of other blocks may be possible.

As described above, when the optimal size and shape of a block is found,mode information (for example, division information, etc.) may begenerated. The mode information may be stored in a bitstream togetherwith information (for example, prediction-related information,transformation-related information, etc.) generated by a component towhich a block belongs and transmitted to a decoder, and may be parsed ina unit of the same level in the decoder and used in an image decodingprocess.

Hereinafter, a division method will be described, and for convenience ofdescription, it is assumed that an initial block is in the form of asquare. However, the initial block may be applied in the same or similarmanner even in the form of a rectangle, but is not limited thereto.

Various methods for block division may be supported, but the presentinvention will focus on tree-based division, and at least one treedivision may be supported. In this case, a quad tree (Quad Tree. QT), abinary tree (BT), a ternary tree (TT), and the like may be supported.When one tree method is supported, it can be referred to as a singletree division and when two or more tree methods are supported, it can bereferred to as a multiple tree method.

The quad-tree division means that a block is divided into two in thehorizontal and vertical direction respectively, the binary tree divisionmeans that a block is divided into two in either the horizontal orvertical direction, and the ternary-tree division means that a block isdivided into three in either the horizontal or vertical direction.

In the present invention, if a block before division is M×N, it isassumed that the block is divided into four M/2×N/2 in the case of thequad-tree division, the block is divided into M/2×N or M×N/2 in the caseof the binary-tree division, and the block is divided intoM/4×N/M/2×N/M/4×N or M×N/4/M×N/2/M×N/4 in the case of the ternary-treedivision. However, the division result is not limited to the above case,and examples of various modifications may be possible.

Depending on encoding/decoding settings, one or more of tree divisionmethods may be supported. For example, quad tree division can besupported, quad tree division and binary tree division can be supported,quad tree division and ternary tree division can be supported, or quadtree division, binary tree division, and ternary tree division can besupported.

The above example is an example of a case where the basic divisionmethod is the quad tree, and binary tree division and ternary treedivision are included in additional division modes depending on whetherother trees are supported, but various modifications may be possible.Herein, information on whether other trees are supported(bt_enabled_flag, tt_enabled_flag, bt_tt_enabled_flag, etc. it may havea value of 0 or 1, if 0: not supported, and if 1: supported) may bedetermined implicitly according to encoding/decoding settings, or may beexplicitly determined in a unit of a sequence, picture, slice, tile, orthe like.

The division information may include information on whether to divide(tree_part_flag. or, qt_part_flag, bt_part_flag, tt_part_flag,bt_tt_part_flag. it may have a value of 0 or 1, and if 0: not dividedand if 1: divided). In addition, information on a division direction(dir_part_flag. or, bt_dir_part_flag, tt_dir_part_flag,bt_tt_dir_part_flag. it may have a value of 0 or 1, if 0: <horizontal>and if 1: <vertical>) may be added according to a division method(binary tree, ternary tree), which may be information that can begenerated when division is performed.

When multiple tree division is supported, various division informationconfigurations may be possible. The following will be described assumingan example of how division information is configured at one depth level(i.e., recursive division may be possible because the supported divisiondepth is set to one or more, but for convenience of explanation).

As an example (1), information on whether to divide is checked. Herein,if the division is not performed, the division is ended.

If the division is performed, division information for a division type(For example, tree_idx. if 0: QT, if 1: BT, if 2: TT) is checked. Inthis case, division direction information is additionally checkedaccording to the selected division type, and the process proceeds to thenext step (if additional division is possible for reasons such as whenthe division depth has not reached the maximum value, etc., the divisionis restarted, and if division is not possible, the division is ended.)

As an example (2), information on whether to divide for some treemethods (QT) is checked, and the process goes to the next step. Herein,if division is not performed, information on whether to divide for sometree methods (BT) is checked. Herein, if division is not performed,information on whether to divide for some tree methods (TT) is checked.Herein, if division is not performed, the division process is ended.

If division of some tree method (QT) is performed, the process goes tothe next step. In addition, if division of some tree methods (BT) isperformed, division direction information is checked and the processgoes to the next step. In addition, if division of some tree methods(TT) is performed, division direction information is checked and theprocess goes to the next step.

As an example (3), information about whether to divide for some treemethods (QT) is checked. Herein, if division is not performed,information on whether to divide for some tree methods (BT and TT) ischecked. Herein, if division is not performed, the division process isended.

If division of some tree methods (QT) is performed, the process goes tothe next step. In addition, if division of some tree methods (BT and TT)is performed, division direction information is checked and the processgoes to the next step.

The above example may be a case where the priority of tree divisionexists (examples 2 and 3) or does not exist (example 1), but examples ofvarious modifications may be possible. In addition, in the aboveexample, the division of the current stage is an example for explainingthe case that is not related to a division result of a previous stage,but it may also be possible to set the division of the current stagedepending on the division result of the previous stage.

For example, in the case of Examples 1 to 3, if division of some treemethods (QT) is performed in a previous step and the process is passedto a current step, division of the same tree methods (QT) may besupported in the current step.

On the other hand, if division of some tree methods (QT) is notperformed in a previous step and division of other tree methods (BT orTT) is performed and the process is passed to a current step, it mayalso be possible that division of some tree methods (BT and TT), exceptfor division of some tree methods (QT), are supported in subsequentsteps including the current step.

In the above case, it means that tree configurations supported for blockdivision may be adaptive, and thus, the above-described divisioninformation configurations may also be configured differently. (assumingthe example to be described later is the third example) That is, in theabove example, if the division of some tree methods (QT) was notperformed in the previous step, the division process may be performedwithout considering some tree methods (QT) in the current stage. Inaddition, it may be configured by removing division informationregarding related tree methods (for example, information about whetherto divide, information about a division direction, etc. in this example<QT>, information about whether to divide).

The above example is for adaptive division information configuration fora case where block division is allowed (for example, a block size iswithin the range between the maximum and minimum values, a divisiondepth of each tree method does not reach the maximum depth <alloweddepth>, etc.), and adaptive division information configuration may bepossible even when block division is limited (for example, a block sizeis not in the range between the maximum and minimum values, a divisiondepth of each tree method reaches the maximum depth, etc.).

As already mentioned, tree-based division in the present invention canbe performed using a recursive method. For example, when a division flagof a coding block having a division depth k is 0, encoding of the codingblock is performed in the coding block having the division depth k, andwhen the division flag of the coding block having the division depth kis 1, encoding of the coding block is performed in N sub-coding blockshaving a division depth of k+1 according to a division method (wherein Nis an integer of 2 or more such as 2, 3, 4).

The sub-coding block may be set again as a coding block (k+1) anddivided into sub-coding blocks (k+2) through the above process, and sucha hierarchical division method may be determined according to divisionsettings such as a division range and a division allowable depth.

Herein, a bitstream structure for representing division information canbe selected from one or more scan methods. For example, a bitstream ofdivision information may be configured based on a division depth order,or the bitstream of the division information may be constructed based onwhether division is performed.

For example, in the case of a division depth order criterion, it means amethod of obtaining division information at a current level depth basedon an initial block and obtaining division information at the next leveldepth. In addition, in the case of a criterion on whether division isperformed, it means a method of preferentially acquiring additionaldivision information in a block divided based on an initial block, andother additional scanning methods may be considered. In the presentinvention, it is assumed that a bitstream of division information isconfigured based on whether division is performed.

As described above, various cases of block division have been described,and a fixed or adaptive setting for block division may be supported.

Herein, a setting related to block division may explicitly includerelated information in a unit such as a sequence, a picture, a slice,and a tile. Alternatively, the block division setting may be determinedimplicitly according to encoding/decoding settings, wherein theencoding/decoding settings may be configured according to one or acombination of two or more of various encoding/decoding elements such asan image type (I/P/B), color component, division type, and divisiondepth.

In an image encoding method according to an embodiment of the presentinvention, intra prediction may be configured as follows. The intraprediction of the prediction unit may comprise constructing a referencepixel, generating a prediction block, determining a prediction mode, andencoding a prediction mode. In addition, the image encoding apparatusmay be configured to comprise a reference pixel configuration unit, aprediction block generation unit, and a prediction mode encoding unitthat implement a reference pixel configuration step, a prediction blockgeneration step, a prediction mode determination step, and a predictionmode encoding step. Some of the above-described processes may be omittedor other processes may be added. In addition, it may be changed in anorder other than the order described above.

FIG. 4 is an exemplary diagram illustrating an intra prediction modeaccording to an embodiment of the present invention.

Referring to FIG. 4, it is explained assuming that 67 prediction modesare configured as a prediction mode candidate group for intra predictionand 65 of which are directional modes and 2 of which are non-directionalmodes (DC, Planar). However, it may not be limited to this, and variousconfigurations may be possible. Herein, the directional mode may bedivided into slope (e.g., dy/dx) or angle information (degree). Inaddition, all or part of the prediction modes may be included in aprediction mode candidate group of a luminance component or achrominance component, and other additional modes may be included in theprediction mode candidate group.

In the present invention, a direction of a directional mode may mean astraight line, and a curved directional mode may also be configured as aprediction mode. In addition, in the case of a non-directional mode, itmay include a DC mode for obtaining a prediction block with the average(or weighted average, etc.) of pixels of adjacent neighboring blocks(for example, left, top, top left, top right, and bottom left blocks) ofthe current block, and a Planar mode for obtaining a prediction blockthrough linear interpolation of pixels of the neighboring blocks, etc.

Herein, in the DC mode, a reference pixel used for generating theprediction block may be obtained from blocks grouped in variouscombinations such as left, top, left+top, left+bottom left, top+topright, left+top+bottom left+top right, etc. In addition, a blockposition at which the reference pixel is obtained may be determinedaccording to encoding/decoding settings defined by an image type, colorcomponent, block size/shape/position, and the like.

Herein, in the planar mode, a pixel used for generating a predictionblock may be obtained in a region composed of a reference pixel (e.g.,left, top, top left, top right, bottom left, or the like) and a regionnot composed of a reference pixel (e.g., right, bottom, bottom right,etc.). In the case of a region not composed of a reference pixel (thatis, it is not encoded), it can be obtained implicitly by using one ormore pixels (for example, copy as it is, weighted average, etc.) in aregion composed of a reference pixel, or information on at least onepixel in the region not composed of the reference pixel may be generatedexplicitly. Therefore, a prediction block may be generated using theregion composed of the reference pixel and the region not composed ofthe reference pixel as described above.

FIG. 5 is a conceptual diagram illustrating intra prediction for adirectional mode and a non-directional mode according to an embodimentof the present invention.

Referring to (a) of FIG. 5, intra prediction according to modes in thevertical (5 a), horizontal (5 b), and diagonal (5 c to 5 e) directionsis illustrated. Referring to (b) of FIG. 5, intra prediction accordingto the DC mode is illustrated. Referring to (c) of FIG. 5, intraprediction according to the planar mode is illustrated.

In addition to the above description, an additional non-directional modemay be included. In the present invention, linear directional modes andnon-directional modes of a DC mode and a planar mode are mainlydescribed, but a change to other cases may also be applied.

FIG. 4 may be prediction modes that are fixedly supported regardless ofa size of a block. In addition, prediction modes supported according toa block size may be different from FIG. 4.

For example, the number of prediction mode candidate groups may beadaptive (e.g., an angle between the prediction modes is equally spaced,but the angle is set differently. the number of the directional modes is9, 17, 33, 65, 129, etc.), or the number of prediction mode candidategroups may be fixed, but may have different configurations (e.g.,directional mode angle, non-directional type, etc.).

In addition, FIG. 4 may be prediction modes that are fixedly supportedregardless of a block type. In addition, prediction modes supportedaccording to a block type may be different from FIG. 4.

For example, the number of prediction mode candidate groups may beadaptive (e.g., set the number of prediction modes derived from thehorizontal or vertical direction depending on the horizontal/verticalratio of the block larger or shorter), or the number of prediction modecandidate groups may be fixed, but may have different configurations(e.g., set the prediction modes derived from the horizontal or verticaldirection depending on the horizontal/vertical ratio of the block morespecifically).

Alternatively, prediction modes of the longer block length may support alarger number, and prediction modes of the shorter block length maysupport a smaller number. In the case of a long block, a prediction modeinterval may support a mode located on the right side of mode 66 (e.g.,a mode having an angle of +45 degrees or more based on the 50th mode,that is, a mode having a number such as 67 to 80) or a mode located onthe left side of mode 2 (e.g., a mode having an angle of −45 degrees ormore based on the 18th mode. that is, a mode having a number such as −1to −14) in FIG. 4. This may be determined according to the ratio of thehorizontal length to the vertical length of the block, and vice versa.

In the present invention, a prediction mode is mainly described as acase where the prediction mode is a fixedly supported prediction mode(regardless of any encoding/decoding element) as shown in FIG. 4, but itmay also be possible to set an adaptively supported prediction modeaccording to encoding settings.

In addition, when classifying prediction modes, horizontal and verticalmodes (18 and 50 modes), and some diagonal modes (Diagonal up right <2>,Diagonal down right <34>, Diagonal down left <66>, etc.) can be astandard, and this may be a classification method performed based onsome directionality (or angle. 45 degrees, 90 degrees, etc.).

In addition, some modes (2 and 66 modes) located at both ends of thedirectional mode may be a mode that is the basis for the prediction modeclassification, which is an example that is possible when the intraprediction mode is configured as illustrated in FIG. 4. That is, when aprediction mode configuration is adaptive, an example in which thereference mode is changed may also be possible. For example, mode 2 canbe replaced by a mode having a number less than or greater than 2 (−2,−1, 3, 4, etc.), or mode 66 can be replaced by a mode having a numberless than or greater than 66 (64, 66, 67, 68, etc.).

In addition, an additional prediction mode for color components may beincluded in a prediction mode candidate group. The following describes acolor copy mode and a color mode as examples of the prediction mode.

(Color Copy Mode)

A prediction mode related to a method of obtaining data for generating aprediction block from regions located in different color spaces may besupported.

For example, a prediction mode for a method of acquiring data forgenerating a prediction block in another color space using a correlationbetween color spaces may be an example of this.

FIG. 6 is a conceptual diagram illustrating intra prediction regarding acolor copy mode according to an embodiment of the present invention.Referring to FIG. 6, the current block C of the current space M mayperform prediction using data of a corresponding region D of a differentcolor space N at the same time t.

In this case, a correlation between color spaces may refer to acorrelation between Y and Cb, Y and Cr, and Cb and Cr when YCbCr istaken as an example. That is, in the case of the chrominance component(Cb or Cr), a reconstructed block of the luminance component Ycorresponding to the current block can be used as a prediction block ofthe current block (chrominance vs. luminance is a default setting of anexample described later). Alternatively, a reconstructed block of somechrominance component (Cb or Cr) corresponding to the current block ofsome chrominance component (Cr or Cb) may be used as a prediction blockof the current block.

Herein, in some color formats (e.g., YCbCr 4:4:4, etc.), a regioncorresponding to the current block may have the same absolute positionin each image. Alternatively, in some color formats (e.g., YCbCr 4:2:0,etc.), relative positions in each image may be the same. Thecorresponding position may be determined according to the ratio of thehorizontal length to the vertical length according to a color format,and corresponding pixels in a color space different from pixels of thecurrent image may be obtained by multiplying or dividing each componentof the coordinates of the current pixel by a ratio of the horizontallength to the vertical length according to a color format.

For convenience of description, the description will be mainly focusedon the case of some color formats (4:4:4), but it should be understoodthat the location of the corresponding region of other color space canbe determined according to the ratio of the horizontal length to thevertical length according to the color format.

In the color copy mode, a reconstructed block of a different color spacemay be used as a prediction block or a block obtained by considering acorrelation between color spaces may be used as a prediction block. Theblock obtained by considering the correlation between color spaces meansa block that can be obtained by performing correction on an existingblock. Specifically, in the formula of {P=a*R+b}, a and b mean valuesused for correction, and R and P mean values obtained in different colorspaces and prediction values of the current color space, respectively.Herein, P means a block obtained by considering the correlation betweencolor spaces.

In this example, it is assumed that data obtained by using a correlationbetween color spaces is used as a prediction value of the current block,but it may also be possible when the data is used as a correction valueapplied to the prediction value of the existing block. That is, theprediction value of the current block can be corrected using a residualvalue of a different color space.

In the present invention, it is assumed on the assumption of the formercase, but the present invention may not be limited thereto, and the sameor changed application to a case that the data is used as a correctionvalue may be applicable.

In the color copy mode, whether to support it explicitly or implicitlymay be determined according to encoding/decoding settings. Herein, theencoding/decoding settings may be defined according to one or acombination of two or more of an image type, color component, blockposition/size/shape, and block width/length ratio. In addition, in anexplicit case, related information may be included in a unit of asequence, picture, slice, or tile.

In addition, depending on encoding/decoding settings, whether the colorcopy mode is supported may be determined implicitly in some cases, andrelated information may be explicitly generated in some cases.

In the color copy mode, correlation information (a, b, etc.) betweencolor spaces may be explicitly generated or implicitly obtainedaccording to encoding/decoding settings.

In this case, a region to be compared (or referenced) to obtaincorrelation information may be the current block (C in FIG. 6) and acorresponding region (D in FIG. 6) of a different color space.Alternatively, it may be an adjacent region (left, top, top left, topright, bottom left blocks, etc., of C in FIG. 6) of the current blockand an adjacent region (left, top, top left, top right, bottom leftblocks, etc., of D in FIG. 6) of a corresponding region of a differentcolor space.

In the above description, in the former case, since correlationinformation must be obtained directly using data of a blockcorresponding to the current block, it may correspond to an example ofexplicitly processing related information. That is, it may be a case inwhich correlation information should be generated because the data ofthe current block has not yet been coded. In the latter case, since thecorrelation information can be obtained indirectly using data of anadjacent region of a block corresponding to an adjacent region of thecurrent block, this may correspond to an example of implicitlyprocessing related information.

In summary, in the former case, correlation information is obtained bycomparing the current block and the corresponding block, and in thelatter case, correlation information is obtained by comparing regionsadjacent to the current block and the corresponding block, respectively.In addition, data obtained by applying correlation information to thecorresponding block may be used as a prediction pixel of the currentblock.

Herein, in the former case, the correlation information can be encodedas it is, or the correlation information obtained by comparing adjacentregions can be used as a predictive value to encode information aboutthe difference. The correlation information may be information that canoccur when a color copy mode is selected as a prediction mode.

Herein, the latter case can be understood as an example of an impliedcase that there is no additionally generated information except that acolor copying mode is selected as an optimal mode in the prediction modecandidate group. That is, this may be an example possible under aconfiguration in which one correlation information is supported.

In a setting in which two or more correlation information is supported,selection information for the correlation information may be required inaddition to the color copy mode being selected as the optimal mode. Asin the above example, a configuration in which an explicit case and animplicit case are mixed may also be possible according toencoding/decoding settings.

In the present invention, a description will be given focusing on thecase where correlation information is obtained indirectly, and theobtained correlation information may be N or more (N is an integer of 1or more such as 1, 2, 3). Setting information on the number ofcorrelation information may be included in a unit such as a sequence,picture, slice, tile, and the like. It should be understood that in someof the examples described below, when two or more correlationinformation is supported, it may have the same meaning as when two ormore color copy modes are supported.

FIG. 7 is an exemplary diagram illustrating a corresponding block ofeach color space and a region adjacent thereto in relation to a colorcopy mode according to an embodiment of the present invention. Referringto FIG. 7, examples of pixel-to-pixel correspondence (p and q) in thecurrent color space (M) and the different color space (N) are shown, andit may be understood as a case where it may occur in some color formats(4:2:0). In addition, the corresponding relationship (7 a) for obtainingcorrelation information and the corresponding relationship (7 b) forapplying a prediction value can be confirmed.

The following continues the description related to obtaining correlationinformation in a color copy mode. In order to obtain correlationinformation, pixel values of pixels in a predetermined region (all orpart of regions adjacent to the current block and a block correspondingto the current block) of each color space may be compared (or used),(i.e., 1:1 pixel value comparison process is performed). In this case,the pixel value to be compared can be obtained based on a correspondingpixel position in each color space. The pixel value may be a valuederived from at least one pixel in each color space.

For example, in some color formats (4:4:4), a pixel value of one pixelin a chrominance space and a pixel value of one pixel in a luminancespace may be used as pixel values corresponding to a correlationinformation acquisition process. Alternatively, in some color formats(4:2:0), a pixel value of one pixel in the chrominance space and a pixelvalue derived from one or more pixels in the luminance space may be usedas pixel values corresponding to the correlation information acquisitionprocess.

Specifically, in the former case, p [x, y] of a chrominance space can becompared with q [x, y] of a luminance space. In this case, as a pixelvalue, a brightness value of one pixel may be used as it is. In thelatter case, p [x, y] in the chrominance space may be compared with q[2x, 2y], q [2x, 2y+1], q [2x+1, 2y], q [2x+1, 2y+1], etc. in theluminance space.

Herein, since the 1:1 pixel value comparison has to be performed, in thecase of the luminance space, one of a plurality of pixels may be used asa value for comparing a pixel value of a chrominance pixel. That is, abrightness value of one pixel among the plurality of pixels is used asit is. Alternatively, one pixel value may be derived from two or morepixels (two to four) among the plurality of pixels and used as a valueto be compared. That is, a weighted average (weights may be equallyallocated or non-uniformly allocated to each pixel) may be applied totwo or more pixels.

When a plurality of corresponding pixels exist as in the above example,a pixel value of a predetermined pixel or a pixel value derived from twoor more pixels may be used as a value to be compared. In this case, oneof the two methods for deriving a pixel value to be compared in eachcolor space according to encoding/decoding settings may be used alone orin combination.

The following may be a description on the assumption that a pixel valueof one pixel is used for comparison in a current color space, and one ormore pixels in other color spaces can be used to derive a pixel value.For example, assume that the color format is YCbCr 4:2:0, the currentcolor space is a chrominance space, and the other color space is aluminance space. The method for deriving the pixel value will bedescribed focusing on different color spaces.

For example, it may be determined according to a shape of a block (ratioof the horizontal length to the vertical length). As a detailed example,p [x, y] of a chrominance space adjacent to the longer side of thecurrent block (or a block to be predicted) may be compared with q [2x,2y] of a luminance space, and p [x, y] of the chrominance space adjacentto the shorter side can be compared with the average of q [2x, 2y] and q[2x+1, 2y] in the luminance space.

In this case, adaptive settings such as the above may be applied to someblock types (rectangular) regardless of a ratio of the horizontal lengthto the vertical length or may be applied only when the ratio of thehorizontal length to the vertical length is greater than or equalto/greater than a certain ratio (k:1 or 1:k. k is 2 or more, e.g. 2:1,4:1, etc.).

For example, it may be determined according to a size of a block. As adetailed example, when a size of the current block is greater than orequal to/greater than a certain size (M×N. For example, 2^(m)×2^(n)where m and n are integers greater than or equal to 1, such as 2 to 6),p [x, y] of a chrominance space can be compared with q [2x+1, 2y] of aluminance space, and when it is less than or equal to/less than acertain size, p [x, y] of the chrominance space can be compared with theaverage of q [2x, 2y], q [2x, 2y+1] of the luminance space.

Herein, one or more of the boundary values for size comparison may bepresent, or adaptive settings such as two or more (M₁×N₁, M₂×N₂, etc.)may be possible.

The above examples are some cases that can be considered in terms ofcomputational quantity, and examples of various modifications arepossible, including cases opposite to the above examples.

For example, it may be determined according to a position of a block. Asa detailed example, when the current block is located inside a presetregion (assuming the largest coding block in this example), p [x, y] ofa chrominance space can be compared with the average of q [2x, 2y], q[2x+1, 2y], q [2x, 2y+1], q [2x+1, 2y+1] of a luminance space, and whenlocated at the boundary of the preset region (assuming the top leftboundary in this example), p [x, y] of the chrominance space may becompared with q [2x+1, 2y+1] of the luminance space. The preset regionmay mean a region set based on a slice, tile, block, or the like.Specifically, it can be obtained based on an integer multiple of aslice, tile, and the maximum coding/prediction/transformation block.

As another example, when a current block is located at some boundary ofa region (assuming an top boundary in this example), P [x, y] of thechrominance space adjacent to some boundary (top) can be compared with q[2x+1, 2y+1] of the luminance space, and P [x, y] of the chrominancespace adjacent to the interior (left) of the region may be compared withthe average of q [2x, 2y], q [2x+1, 2y], q [2x, 2y+1], q [2x+1, 2y+1].

The above examples are some cases that can be considered in terms ofmemory, and examples of various modifications are possible includingcases opposite to the above examples.

Through the above-described examples, various cases of pixel valuederivation compared in each color space have been described. As in theabove example, a pixel value derivation setting for obtaining thecorrelation information may be determined in consideration of variousencoding/decoding elements as well as a size/shape/position of a block.

Through the above example, a case where one or two reference pixel linesof the block corresponding to the current block is used respectively asa region to be compared for obtaining correlation information has beendescribed. That is, in the case of YCbCr 4:4:4, one reference pixel lineis used respectively, and in other formats, in some color spaces, itmeans a case in which two reference pixel lines are used as in somecolor spaces <color N> in FIG. 7. In addition, it may not be limited tothis, and examples of various modifications may be possible.

The following will be described with focus on a reference pixel line inthe current color space, and it should be understood that in other colorspaces, a reference pixel line may be determined according to a colorformat. That is, the same number of reference pixel lines may be used,or twice the number of reference pixel lines may be used.

In a color copying mode of the present invention, k reference pixellines (where k is an integer of 1 or more such as 1 and 2) may be used(or compared) for obtaining correlation information. In addition, kreference pixel lines may be used fixedly or adaptively. In thefollowing, various examples of setting the number of reference pixellines will be described.

For example, it may be determined according to a shape of a block (ratioof the horizontal length to the vertical length). As a detailed example,two reference pixel lines adjacent to the longer side of the currentblock may be used, and one reference pixel line adjacent to the shorterside of the current block may be used.

In this case, the above may be applied to some block types (rectangular)regardless of the ratio of the horizontal length to the vertical length,or may be applied only when the horizontal/vertical length ratio isgreater than or equal to a certain ratio (k:1 or 1:k. k is 2 or more,e.g. 2:1, 4:1, etc.). In addition, there are two or more boundary valuesfor the ratio of the horizontal length to the vertical length, in thecase of 2:1 or 1:2, two reference pixel lines adjacent to the longerside (or the shorter side) are used, and in the case of 4:1 or 1:4,extension may be possible, such as using three reference pixel linesadjacent to the longer side (or the shorter side).

In the above example, according to the ratio of the horizontal length tothe vertical length, the longer side (or shorter side) uses s referencepixel lines, and the shorter side (or longer side) uses t referencepixel lines. In this case, it may be an example for a case where s isgreater than or equal to t (i.e., s and t are integers greater than orequal to 1).

For example, it may be determined according to a size of a block. As adetailed example, two reference pixel lines may be used when a size of acurrent block is greater than or equal to/greater than a certain size(M×N. For example, 2^(m)×2 where m and n are integers greater than orequal to 1 such as 2 to 6), and one reference pixel line may be usedwhen the size is less than or equal to/less than a predetermined size.

Herein, one boundary value for size comparison may exist as in the aboveexample, or adaptive settings such as two or more (M₁×N₁, M₂×N₂, etc.)may be possible.

For example, it may be determined according to a position of a block. Asa detailed example, two reference pixel lines may be used when a currentblock is located inside a preset region (derivable from the previousdescription related to obtaining the correlation information. assumingthe largest coding block in this example), and one reference pixel linemay be used when located at the boundary of the preset region (assumingthe top left boundary in this example).

As another example, when a current block is located at some boundary ofthe region (assuming the top boundary in this example), one referencepixel line adjacent to some boundary (top) may be used, and tworeference pixel lines adjacent to the inside (left) of the region may beused.

The above examples are some cases that can be considered in terms ofaccuracy and memory of correlation information, and examples of variousmodifications are possible, including cases opposite to the aboveexamples.

Through the above-described examples, various cases of setting referencepixel lines used to obtain correlation information in each color spacehave been described. As in the above example, a reference pixel linesetting for obtaining correlation information may be determined inconsideration of various encoding/decoding elements as well as asize/shape/position of a block.

The following describes another case for a region to be compared (orreferenced) to obtain correlation information. Pixels adjacent topositions of the left, top, top left, top right, bottom left, or thelike adjacent to the current block in a current color space may betargeted for the region to be compared.

In this case, the region to be compared can be set including all of theblocks in the left, top, top left, top right, and bottom left positions.Alternatively, a reference pixel region may be configured by acombination of blocks at some locations. For example, the region to becompared can be configured by a combination of adjacent blocks such asleft/top/left+top/left+top+top left/left+bottom left/top+topright/left+top left+bottom left/top+top left+top right/left+top+topright/left+top+bottom left block.

In summary, the region to be compared for obtaining correlationinformation may be configured as a predetermined region. Alternatively,it may be configured in various combinations of some regions. That is,the region to be compared may be fixed or adaptively configuredaccording to encoding/decoding settings.

In the following, we will look at various examples of what directionadjacent regions are configured as reference regions of a current blockin a current color space. Herein, it is assumed that in a correspondingblock of a different color space, what direction adjacent regions areconfigured as reference regions is determined according to aconfiguration of a reference region of a current color block. Inaddition, it is assumed that the basic reference region is composed ofthe left and top blocks.

For example, it may be determined according to a shape of the block(horizontal/vertical length ratio). As a detailed example, if thecurrent block is long horizontally, the left, top, and top right blocksmay be set as reference regions, and if the current block is longvertically, the left, top, and bottom left blocks may be set asreference regions.

In this case, the above may be applied to some block shapes(rectangular) regardless of a horizontal/vertical length ratio, or maybe applied only when the horizontal/vertical length ratio is greaterthan or equal to/greater than a certain ratio (k:1 or 1:k. k is 2 ormore, e.g. 2:1, 4:1, etc.). In addition, there are two or more boundaryvalues for the horizontal/vertical length ratio, in the case of 2:1 (or1:2), the left, top, and top right (or the left, top, and bottom left)blocks are set as reference regions, and in the case of 4:1 (or 1:4), anextension such as the top and top right (or the left and bottom left)blocks may be set as reference regions.

For example, it may be determined according to a size of a block. As adetailed example, when a size of a current block is greater than orequal to/greater than a certain size (M×N. For example, 2^(m)×2^(n)where m and n are integers greater than or equal to 1 such as 2 to 6),the left and top blocks may be set as reference regions, and when thesize of the current block is less than or equal to/less than the certainsize, the left, top, and top left blocks may be set as referenceregions.

In this case, one boundary values for size comparison may be present, oradaptive settings such as two or more (M₁×N₁, M₂×N₂, etc.) may bepossible.

For example, it may be determined according to a position of a block. Asa detailed example, when a current block is located inside a presetregion (derivable from the previous description related to obtainingcorrelation information. assuming the largest coding block in thisexample), the left, top, top left, top right, and bottom left blocks areset as reference regions, and when the current block is located at theboundary (assuming the top left boundary in this example) of the presetregion, the left and top blocks are set as reference regions.

As another example, when a current block is located at some boundary(assuming the top boundary in this example) of the region, except forblocks adjacent to some boundary (top boundary), the left and bottomleft blocks adjacent to the region may be set as reference regions. Thatis, the left and bottom left blocks may be set as reference regions.

The above examples are some cases that can be considered in terms ofcomputational quantity, memory, and the like, and examples of variousmodifications are possible, including cases opposite to the aboveexamples.

Through the above-described examples, various cases of setting referenceregions used to obtain correlation information in each color space havebeen described. As in the above example, the reference region settingfor obtaining correlation information may be determined in considerationof various encoding/decoding elements as well as a size/shape/locationof a block.

In addition, the region to be compared may be a pixel adjacent to thecurrent block in the current color space. Herein, all of the referencepixels may be used to obtain correlation information, or some of themmay be used.

For example, when a current block (based on color M in FIG. 7) is ablock having a pixel range of (a, b) to (a+7, b+7) (i.e., 8×8), it isassumed that the region to be compared (because the corresponding blockcan be explained according to a color format, it is omitted) is onereference pixel line of the left and top blocks of the current block.

In this case, all pixels within the range of (a, b−1) to (a+7, b−1) and(a−1, b) to (a−1, b+7) may be included in the region to be compared.Alternatively, (a, b−1), (a+2, b−1), (a+4, b−1), (a+6, b−1), and (a−1,b), (a−1, b+2), (a−1, b+4), (a−1, b+6), which are some pixels in theabove range, may be included. Alternatively, (a, b−1), (a+4, b−1), and(a−1, b), (a−1, b+4), which are some pixels in the above range, may beincluded.

The above example may be applicable for the purpose of reducing theamount of computational quantity required to obtain correlation. Variousencoding/decoding elements such as a size/shape/position of a block maybe considered in a setting of reference pixel sampling of the comparedregion for obtaining correlation information together with the manyexamples already described above. In addition, examples of relatedapplications from the previous example can be derived, and thus detaileddescription is omitted.

Through the various examples described above, various elements(derivation of a corresponding pixel value, number of reference pixellines, reference region direction setting, reference pixel sampling,etc.) influencing acquisition of correlation information were examined.A number of different cases may be possible where the above examplesalone or in combination affect the acquisition of correlationinformation.

The above description can be understood as a preset process forobtaining one correlation information. In addition, as alreadymentioned, one or more correlation information may be supportedaccording to encoding/decoding settings. In this case, two or morecorrelation information may be supported by placing two or more ofpreset settings (that is, a combination of elements affecting theacquisition of the correlation information).

In summary, parameter information based on correlation information canbe derived from an adjacent region of a current block and an adjacentregion of a corresponding block. That is, at least one parameter (e.g.,<a1, b1>, <a2, b2>, <a3, b3>, etc.) may be generated based on thecorrelation information, and it can be used as a value that ismultiplied or added to a pixel of a reconstructed block in a differentcolor space.

The following continues a description of a linear model applied in acolor copy mode. Prediction based on the following linear model can beperformed by applying the parameters obtained through the above process.

[Equation 1]

prec_sample_C(i,j)=a×rec_sample_D(i,j)+b

In the above equation, pred_sample_C means a prediction pixel value of acurrent block in a current color space, and rec_sample_D means areconstructed pixel value of a corresponding block in another colorspace, a and b can be obtained by minimizing regression error between anadjacent region of the current block and an adjacent region of thecorresponding block, and can be calculated by the following equation.

$\begin{matrix}{{a = \frac{{N \times {\Sigma\left( {{D(n)} \times {C(n)}} \right)}} - {\Sigma{D(n)} \times \Sigma{C(n)}}}{{N \times {\sum\left( {{D(n)} \times {C(n)}} \right)}} - {\sum{{D(n)} \times {\sum{C(n)}}}}}}{b = \frac{{\Sigma{C(n)}} - {a \times \Sigma{D(n)}}}{N}}} & \left\lbrack {{Equation}2} \right\rbrack\end{matrix}$

In the above equation, D(n) means an adjacent region of thecorresponding block, C(n) means an adjacent region of the current block,and N means a value (in this example, it is assumed that it is twice theminimum value of the horizontal or vertical length) set based on thehorizontal or vertical length of the current block.

In addition, various methods such as a Simplified Straight-Line Equationfor obtaining correlation information based on the minimum and maximumvalues of adjacent regions of each color space can be used. In thiscase, as a model for obtaining correlation information, one preset modelmay be used, or one of a plurality of models may be selected. Herein,the meaning of selecting one of the plurality of the models means thatmodel information may be considered as encoding/decoding elements forparameter information based on correlation information. That is, when aplurality of parameters are supported, it may mean that the remainingcorrelation information related settings may be classified intodifferent parameter information according to different models forobtaining correlation even though they are the same.

In some color formats (if not 4:4:4), one pixel of a current block maycorrespond to one or more (2, 4, etc.) pixels of a corresponding block.For example, in the case of 4:2:0, p [x, y] in a chrominance space maycorrespond to q [2x, 2y], q [2x, 2y+1], q [2x+1, 2y], q [2x+1, 2y+1,etc. in a luminance space.

For one prediction pixel value, one pixel value may be derived from apixel value (or prediction value) of a predetermined pixel or two ormore pixels among the corresponding plurality of pixels. Depending onencoding/decoding settings, various cases may be possible, and adetailed description thereof will be omitted because a relateddescription can be derived from a section on deriving a correspondingpixel value to obtain correlation information.

(Color Mode)

A prediction mode related to a method of obtaining a prediction mode forgenerating a prediction block from regions located in different colorspaces may be supported.

For example, a prediction mode for a method of obtaining a predictionmode for generating a prediction block in another color space usingcorrelation between color spaces may be an example. That is, a colormode may be a mode that is adaptively determined according to aprediction mode of a block corresponding to a different color space byusing existing prediction directions and methods, rather than having anyspecific prediction direction or prediction method.

In this case, it may be possible that various color modes are obtainedaccording to a block division setting.

For example, in a setting (i.e., when a block division of a luminancecomponent is explicitly determined) in which block division for somecolor components (chrominance) is implicitly determined according to aresult of block division for some color components (luminance), oneblock of some color components (chrominance) may correspond to one blockof some color spaces (luminance). Therefore, (assuming 4:4:4. for otherformats, explanation of this example can be derived depending on theratio of the horizontal length to the vertical length) if a currentblock (chrominance) has a pixel range of (a, b) to (a+m, b+n), even ifit points to any pixel position within the pixel range of (a, b) to(a+m, b+n) of the corresponding block (luminance), since it points toone block, one prediction mode may be obtained in a block including thecorresponding pixel.

Alternatively, in the case where individual block division is supportedaccording to each color component (i.e., block division of each colorspace is explicitly determined), one block of some color components(chrominance) may correspond to one or more blocks of some color spaces(luminance). Therefore, even if the current block (chrominance) has thesame pixel range as the above example, the corresponding block(luminance) may be composed of one or more blocks according to theresult of block division. Therefore, different prediction modes (i.e.,one or more modes) may be obtained from corresponding blocks indicatedby the corresponding pixels according to the position of pixels within apixel range of the current block.

If one color mode is supported by intra prediction mode candidate groupsfor chrominance components, it can be set where to get the predictionmode from the corresponding block.

For example, a prediction mode may be obtained at the location of thecenter-top left-top right-bottom left-bottom right, etc. of acorresponding block. That is, if a prediction mode is obtained in theabove order, but the corresponding block is not available (e.g., theencoding mode is inter, etc.), a prediction mode of a positioncorresponding to the next order can be obtained. Alternatively, aprediction mode having a high frequency (two or more times) in theblocks at the location may be obtained.

Alternatively, when supporting multiple color modes, it is possible toset where to obtain a prediction mode according to the priority.Alternatively, a combination may be possible in which some predictionmodes are obtained according to the priority and some prediction modeshaving high frequencies in blocks at the location are obtained. Herein,the priority is an example, and examples of various modifications may bepossible.

A color mode and a color copy mode may be prediction modes that can besupported for chrominance components. For example, a prediction modecandidate group for chrominance components may be configured includinghorizontal, vertical, DC, planar, diagonal mode, etc. Alternatively, anintra prediction mode candidate group may be configured including thecolor mode and the color copy mode.

That is, a configuration may be directional+non-directional+color modeor directional+non-directional+color copy mode, ordirectional+non-directional+color mode+color copy mod. In addition, amode for additional chrominance components may be included andconfigured.

Whether to support a color mode and a color copy mode may be determinedaccording to encoding/decoding settings, implicit or explicit processingmay be possible in this case. Alternatively, a mixed configuration ofexplicit+implicit processing may be possible. This includes detailedsettings related to the color mode and color copy mode (for example, thenumber of supported modes, etc.), so that implicit or explicitprocessing may be possible.

For example, the related information may be explicitly included in aunit of a sequence, picture, slice, tile, block, or the like, or may bedetermined implicitly according to various encoding/decoding elements(for example, image type, block location, block size, block shape, blockwidth/length ratio, etc.). Alternatively, depending on theencoding/decoding elements, some conditions may be implicitlydetermined, or related information may be explicitly generated in someconditions.

FIG. 8 is an exemplary diagram for explaining a reference pixelconfiguration used for intra prediction according to an embodiment ofthe present invention. A size and shape (M×N) of a prediction block maybe obtained through a block division unit.

Block range information defined as the minimum block and maximum blocksize for intra prediction may include related information in a unit suchas a sequence, picture, slice, tile, etc. In general, the horizontal andvertical lengths are specified (for example, 32×32, 64×64, etc.), sothat the size information may be set, but the size information may alsobe set in the form of the product of the horizontal and verticallengths. For example, when the product of horizontal and vertical is 64,the minimum block size may correspond to 4×16, 8×8, 16×4, or the like.

In addition, by specifying the horizontal and vertical lengths, settingsize information or setting size information in the form of a productmay be mixed and used. For example, for the maximum block size, if theproduct of the horizontal and vertical lengths is 4096 and the maximumvalue of one of the two lengths is 64, 64×64 may correspond to themaximum block size.

As in the above example, in addition to the minimum block and maximumblock size information, block division information is mixed to finallydetermine a size and shape of a prediction block. In the presentinvention, the prediction block must have the product of the horizontaland vertical lengths greater than or equal to s (for example, s is amultiple of 2, such as 16, 32) and one of the horizontal/verticallengths greater than or equal to k (for example, k is 4, 8, etc.). Inaddition, although the horizontal and vertical lengths of the block maybe defined under a setting equal to or less than v and w (e.g., v and ware multiples of 2, such as 16, 32, 64, etc.), respectively. Inaddition, it may not be limited thereto, and various block rangesettings may be possible.

Intra prediction may be generally performed in a unit of a predictionblock, but may be performed in a unit of a coding block, transformblock, or the like according to a setting of the block division unit.After checking block information, the reference pixel configuration unitmay configure a reference pixel used for prediction of a current block.In this case, a reference pixel may be managed through temporary memory(for example, an array. 1st, 2nd array, etc.), generated and removed foreach intra prediction process of a block, and the size of the temporarymemory may be determined according to the configuration of the referencepixel.

In this example, it described assuming that the left, top, top left, topright, and bottom left blocks are used for prediction of a currentblock, but it may not be limited thereto, and a block candidate grouphaving a different configuration may be used for prediction of thecurrent block. For example, a candidate group of neighboring blocks forthe reference pixel may be an example of following a raster or Z scan,and some of the candidate group may be removed according to a scan orderor may be configured including other block candidates (For example,right, bottom, bottom right blocks, etc.).

In addition, if some prediction mode (color copy mode) is supported,some regions of different color spaces can be used for prediction of acurrent block, and it can also be considered as a reference pixel. Theexisting reference pixels (spatial adjacent regions of the currentblock) and the additional reference pixels may be managed as one orseparately (for example, the reference pixel A and the reference pixelB. That is, the reference pixel memory may be separately named as if thetemporary memory is used separately).

For example, the temporary memory of the basic reference pixel may havea size of <2×blk_width+2×blk_height+1> (based on one reference pixelline), and the temporary memory of the additional reference pixel mayhave a size of <blk_width×blk_height> (when 4:4:4)(blk_width/2×blk_height/2 is required when 4:2:0). The temporary memorysize is one example and is not limited thereto.

In addition, it may be managed as a reference pixel including adjacentregions of the current block to be compared (or referenced) to obtaincorrelation information and a corresponding block. That is, additionalreference pixels may be managed according to a color copy mode.

In summary, an adjacent region of a current block may be included as areference pixel for intra prediction of the current block, and acorresponding block of a different color space and its adjacent regionmay be included as a reference pixel according to the prediction mode.For convenience of description, a description will be given mainly ofthe case of a basic reference pixel configuration.

As illustrated in FIG. 8, reference pixels used for prediction of thecurrent block may be configured as adjacent pixels (ref_L, Ref_T,Ref_TL, Ref_TR, Ref_BL in FIG. 8) of the left, top, top left, top right,and bottom left blocks. In this case, the reference pixel is generallycomposed of pixels of the neighboring block closest to the current block(a in FIG. 8 as a reference pixel line), but other pixels (pixels inFIG. 8 and other outer lines) may also be configured in the referencepixel.

Pixels adjacent to a current block may be classified into at least onereference pixel line, and the pixel closest to the current block may beclassified into ref_0 {e.g., pixels having a distance of 1 between aboundary pixel of the current block and the pixel. p(−1, −1) to p(2m−1,−1), p(−1,0) to p(−1,2n−1)}, the next adjacent pixel to ref_1 {e.g., thedistance between the boundary pixel of the current block and the pixelis 2. p(−2, −2) to p(2m, −2), p(−2, −1) to p (−2, 2n)}, and the nextadjacent pixel to ref_2 {e.g., the distance between the boundary pixelof the current block and the pixel is 3. p(−3, −3) to p(2m+1, −3), p(−3,−2) to p(−3, 2n+1)}. That is, it can be classified as a reference pixelline according to a pixel distance adjacent to the boundary pixel of thecurrent block.

Herein, the number of reference pixel lines supported may be N or more,and N may be an integer of 1 or more, such as 1 to 5. In this case, itis generally included in the reference pixel line candidate groupsequentially from the reference pixel line closest to the current block,but is not limited thereto. For example, the candidate groups may besequentially configured as <ref_0, ref_1, ref_2> when N is 3, or it mayalso be possible that the candidate group is configured with aconfiguration that excludes closest reference pixel lines or anon-sequential configuration such as <ref_0, ref_1, ref 3>, <ref_0,ref_2, ref 3>, <ref_1, ref_2, ref_3>.

Prediction may be performed using all reference pixel lines in thecandidate group, or prediction may be performed using some referencepixel lines (one or more).

For example, one of a plurality of reference pixel lines may be selectedaccording to encoding/decoding settings, and intra prediction may beperformed using the reference pixel line. Alternatively, two or more ofthe plurality of reference pixel lines may be selected to use thecorresponding reference pixel line (for example, a weighted average isapplied to the data of each reference pixel line) to perform intraprediction.

Herein, the reference pixel line selection may be determined implicitlyor explicitly. For example, in the implicit case, it means that it isdetermined according to encoding/decoding settings defined according toone or a combination of two or more elements such as an image type,color component, and a size/shape/position of a block. In addition, theexplicit case means that reference pixel line selection information maybe generated in a unit such as a block.

Although the present invention mainly describes a case in which intraprediction is performed using the closest reference pixel line, itshould be understood that various embodiments described below may beapplied to the same or similar application when multiple reference pixellines are used.

The reference pixel configuration unit of the intra prediction of thepresent invention may include a reference pixel generation unit, areference pixel interpolation unit, and a reference pixel filter unit,and may include all or part of the above configuration.

The available reference pixel and the unavailable reference pixel may beclassified by checking the availability of the reference pixel in thereference pixel configuration unit. Herein, it is determined that theavailability of the reference pixel is unavailable when at least one ofthe following conditions is satisfied.

For example, if any of the below cases are satisfied, it may bedetermined as unavailable: if it is located outside the pictureboundary, if it does not belong to the same division unit (e.g., unitsthat cannot be referenced to each other, such as slices and tiles,except for division units with a referenceable characteristic) as thecurrent block, and if encoding/decoding is not completed. That is, whennone of the above conditions are satisfied, it can be determined asavailable.

In addition, it is possible to limit the use of the reference pixelbased on encoding/decoding settings. For example, the use of thereference pixel may be limited according to whether limited intraprediction (e.g., constrained_intra_pred_flag) is performed, even if itis determined to be usable according to the above conditions. Thelimited intra prediction may be performed when error-resistantencoding/decoding is performed on an external factor such as acommunication environment, or when a block referenced and reconstructedfrom another image is prohibited to be used as a reference pixel.

When the limited intra prediction is disabled (e.g., I picture type. orconstrained_intra_pred_flag=0 in P or B picture type), all of thereference pixel candidate blocks may be available.

Alternatively, when the limited intra prediction is activated (forexample, constrained_intra_pred_flag=1 in P or B image type), thereference pixel candidate block will be assumed as a condition fordetermining whether to use the reference pixel according to an encodingmode (intra or inter). In addition, the above condition may bedetermined according to various other encoding/decoding elements.

Since the reference pixel is composed of one or more blocks, when anavailability of the reference pixel is confirmed and classified, it canbe classified into three cases: <all usable>, <some usable>, and <notall usable>. In all cases other than the case of <all usable>, areference pixel at an unavailable candidate block position may be filledor generated.

When a reference pixel candidate block is available, a pixel at acorresponding position may be included in a reference pixel memory of acurrent block. In this case, the pixel data may be copied as it is ormay be included in the reference pixel memory through processes such asreference pixel filtering and reference pixel interpolation. Inaddition, when the reference pixel candidate block is unavailable, thepixel obtained through the reference pixel generation process may beincluded in the reference pixel memory of the current block.

The following is an example of generating a reference pixel at anunusable block location using various methods.

For example, a reference pixel may be generated using an arbitrary pixelvalue. Herein, the arbitrary pixel value may be one pixel value (e.g.,the minimum value, maximum value, median value, etc. of the pixel valuerange) belonging to the pixel value range (for example, a pixel valuerange based on a bit depth or a pixel value range according to a pixeldistribution in a corresponding image). Specifically, it may be anexample applied when all of the reference pixel candidate blocks areunavailable.

Alternatively, a reference pixel may be generated from a region in whichimage encoding/decoding is completed. Specifically, a reference pixelmay be generated from at least one usable block adjacent to an unusableblock. In this case, at least one of methods such as extrapolation,interpolation, and copying can be used.

After reference pixel configuration is completed in the reference pixelinterpolation unit, a reference pixel in a decimal unit may be generatedthrough linear interpolation of the reference pixel. Alternatively, thereference pixel interpolation process may be performed after thereference pixel filter process described below.

In this case, the interpolation process may not be performed in the caseof horizontal, vertical, some diagonal modes (e.g., modes of 45 degreesdifference in vertical/horizontal such as Diagonal top right, Diagonalbottom right, Diagonal bottom left mode. corresponding to mode 2, 34,and 66 in FIG. 4), non-directional mode, color copy mode, etc., and theinterpolation process may be performed in other modes (other diagonalmodes).

A pixel position (i.e., which decimal unit is interpolated. it isdetermined from ½ to 1/64, etc.) where interpolation is performed may bedetermined according to a prediction mode (e.g., directionality of theprediction mode, dy/dx, etc.) and positions of a reference pixel and aprediction pixel. In this case, one filter (e.g., assume a filter withthe same equation used to determine filter coefficients or a length offilter taps. however, it is assumed that only the coefficients areadjusted according to the precision <for example, 1/32, 7/32, 19/32> ofa decimal unit) may be applied regardless of the precision of a decimalunit, or one of a plurality of filters (e.g., assume a filter with aseparate equation used to determine filter coefficients or a length offilter tabs) may be selected and applied according to the decimal unit.

The former case may be an example of using an integer unit pixel as aninput for interpolation of a decimal unit pixel, and the latter case maybe an example of different input pixels step by step (for example, inthe case of a ½ unit, integer pixels are used. in the case of a ¼ unit,integer and ½ unit pixels are used, etc.), but it is not limitedthereto, and in the present invention, the former case will be mainlydescribed.

For reference pixel interpolation, fixed filtering or adaptive filteringmay be performed, and this may be determined according toencoding/decoding settings (for example, one or a combination of two ormore of an image type, a color component, a block position/a size/ashape, a block width/height ratio, a prediction mode, etc.).

The fixed filtering may perform reference pixel interpolation using onefilter, and the adaptive filtering may perform reference pixelinterpolation using one of a plurality of filters.

Herein, in the case of the adaptive filtering, one of a plurality offilters may be determined implicitly or explicitly according toencoding/decoding settings. Herein, a type of a filter can be composedof a 4-tap DCT-IF filter, a 4-tap cubic filter, a 4-tap Gaussian filter,a 6-tap Wiener filter, and an 8-tap Kalman filter. In addition, it mayalso be possible to define different filter candidate groups supportedaccording to color components (for example, some types of filters arethe same or different, and a length of filter tabs is short or long,etc.).

The reference pixel filter unit may perform filtering on a referencepixel for the purpose of improving prediction accuracy by reducingremaining deterioration through an encoding/decoding process. The filterused in this case may be a low-pass filter, but is not limited thereto.

Whether filtering is applied may be determined according toencoding/decoding settings (which can be derived from the abovedescription). In addition, when filtering is applied, fixed filtering oradaptive filtering may be applied.

The fixed filtering means that reference pixel filtering is notperformed or reference pixel filtering is applied using one filter. Theadaptive filtering means that whether filtering is applied is determinedaccording to encoding/decoding settings, and if there are two or moresupported filter types, one of them can be selected.

In this case, a plurality of filters classified by various filtercoefficients such as 3-tap filter like [1, 2, 1]/4, 5-tap filter like[2, 3, 6, 3, 2]/16, etc., filter tap lengths, and the like as the filtertype may be supported.

The reference pixel interpolation unit and the reference pixel filterunit introduced in the reference pixel configuration step may benecessary components for improving prediction accuracy. The twoprocesses may be independently performed, but a configuration in whichthe two processes are mixed may also be possible.

The prediction block generation unit may generate a prediction blockaccording to at least one prediction mode, and use a reference pixelbased on the prediction mode. In this case, depending on the predictionmode, the reference pixel may be used in a method (directional mode)such as extrapolation, and may be used in a method (non-directionalmode) such as interpolation, average (DC), or copy.

FIG. 9 is a conceptual diagram illustrating a block adjacent to a targetblock for intra prediction according to an embodiment of the presentinvention. Specifically, the left side of FIG. 9 represents a blockadjacent to the current block in the current color space, and the rightside represents a corresponding block in another color space.

The following describes reference pixels used according to theprediction mode.

In the case of the directional mode, reference pixels of the bottom leftand left blocks (Ref_BL, Ref_L in FIG. 9) may be used for modes betweenhorizontal and some diagonal modes (diagonal up right) (modes 2 to 17 inFIG. 4), reference pixels of the left block may be used for thehorizontal mode, reference pixels of the left, top left, and top blocks(Ref_L, Ref_TL, Ref_T in FIG. 9) may be used for modes betweenhorizontal and vertical modes (modes 19 to 49 in FIG. 4), referencepixels of the top block (Ref_L in FIG. 9) may be used for the verticalmode, and reference pixels of the top and top right blocks (Ref_T,Ref_TR in FIG. 9) may be used for modes between the vertical and somediagonal modes (Diagonal down left) (modes 51 to 66 in FIG. 4).

In addition, in the case of the non-directional mode, reference pixelslocated in one or more of the blocks of the bottom left, left, top left,top, and top right blocks (Ref_BL, Ref_L, Ref_TL, Ref_T, Ref_TR in FIG.9) may be used. For example, it can be used for intra prediction in theform of various reference pixel combinations such as left, top,left+top, left+top+top left, left+top+top left+top right and bottomleft, etc., and this may be determined according to the non-directionalmode (DC, Planar, etc.). In the example to be described later, it isassumed that left+top blocks are used in the DC mode, andleft+top+bottom left+top right blocks are used for prediction as areference pixel in the planar mode.

In addition, in the case of a color copy mode, a reconstructed block(Ref_C in FIG. 9) of another color space can be used as a referencepixel. In the example described later, it is assumed that a blockcorresponding to the current block is used for prediction as referencepixels.

In this case, reference pixels used for intra prediction may beclassified into a plurality of concepts (or units). For example,reference pixels used for intra prediction may be classified into one ormore categories, such as a first reference pixel and a second referencepixel. In this example, it is assumed that a reference pixel isclassified into two categories.

In this case, the first reference pixel may be a pixel used directly forgenerating prediction values of a current block, and the secondreference pixel may be a pixel used indirectly for generating predictionvalues of the current block.

Alternatively, the first reference pixel may be a pixel used to generateprediction values of all pixels of the current block, and the secondreference pixel may be a pixel used to generate prediction values ofsome pixels of the current block.

Alternatively, the first reference pixel may be a pixel used to generatethe first prediction value of the current block, and the secondreference pixel may be a pixel used to generate the second predictionvalue of the current block.

Alternatively, the first reference pixel may be a pixel obtained byapplying pre-processing (e.g., reference pixel filtering, etc.), and thesecond reference pixel may be a pixel obtained by not applyingpre-processing.

Alternatively, the first reference pixel may be a pixel located in aregion positioned at a starting point of a prediction direction of acurrent block (e.g., in a vertical mode, a top block corresponds to astarting point of a prediction direction), and the second referencepixel may be a pixel located in a region (e.g., may or may not belocated at a starting point of a prediction direction) locatedregardless of a prediction direction of the current block.

The first reference pixel, the second reference pixel, and the like maybe data necessary for intra prediction. In detail, the first referencepixel may be essential data that is basically required for intraprediction, and the other reference pixels may be optional data thathelps to improve prediction accuracy.

In the present invention, the prediction block generation process may bebased on using the first reference pixel, but it may also be possible toperform prediction using additional reference pixels. In this case,performing prediction using additional reference pixels may be referredto as prediction block (or pixel) correction.

In the present invention, the prediction block generation unit may beconfigured to further include a prediction block correction unit.Herein, the prediction block correction unit may be regarded as a stepof performing correction on the prediction block according to acorrection setting. In this case, the correction setting may bedetermined by a prediction mode, block information, and the like. Indetail, it may be a configuration that the prediction block correctionunit follows after the prediction block generation unit, but forconvenience of description, assume that the prediction block generationunit includes the prediction block correction unit.

The reference pixel described through the above-described example may bethe first reference pixel, and additionally, the second reference pixelmay be involved in generating a prediction block. The following is adescription of a reference pixel used according to a prediction mode.

In the directional mode, the second reference pixel may be used inaddition to the first reference pixel. Specifically, the top left, topand top right blocks may be additionally used in modes between thehorizontal and some diagonal modes (Diagonal up right), and the bottomleft, top left, top and top right blocks may be additionally used in thehorizontal mode. The bottom left and top right blocks may beadditionally used in modes between the horizontal and vertical modes,and the bottom left, left, top left and top right blocks may beadditionally used in the vertical mode. The bottom left, left, and topleft blocks may be additionally used in modes between vertical and somediagonal modes (Diagonal down left).

In addition, in the non-directional mode, the second reference pixel maybe used in addition to the first reference pixel. Specifically, in theDC mode, the bottom left, top left, and top right blocks may beadditionally used, and in the planar mode, the top left block may beadditionally used.

In addition, in the case of a color copy mode, the second referencepixel may be used in addition to the first reference pixel. In detail,the left, top, top left, top right, and bottom left blocks of thecurrent block may be additionally used. Alternatively, the left, right,top, bottom, top left, top right, bottom left, and bottom right blocksof a corresponding block of a different color space may be additionallyused.

In the above description, the adjacent region of the current block andthe adjacent region of the corresponding block may be regions referencedto obtain correlation information of the color copy mode, and may beregions that are not directly involved in prediction of the currentblock. However, correction is performed for the purpose of improving theaccuracy of prediction, which means that the region can be further usedfor prediction of the current block.

For example, since deterioration of block boundaries remains in the dataof other color spaces, before the post-processing filter is applied,correction may be performed for the purpose of improving accuracy of aprediction block. Alternatively, even if it is obtained as theprediction data of the current block, since the discontinuouscharacteristics may be strong with an adjacent region of the currentblock, correction may be performed for the purpose of improving accuracyof a prediction block.

In the intra prediction of the present invention, the second referencepixel may be considered to include not only an encoded/decoded pixel butalso a pixel in the current block (a prediction pixel in this example).That is, the first prediction value may be a pixel used to generate thesecond prediction value. In the present invention, a description will begiven focusing on an example in which the encoded/decoded pixels areconsidered as the second reference pixel, but it may not be limitedthereto, and an example of modification using pixels in whichencoding/decoding is not completed may also be possible.

Generating or correcting a prediction block using a plurality ofreference pixels may be a case performed for the purpose of compensatingfor the disadvantages of the existing prediction mode.

For example, the directional mode is a mode used for the purpose ofperforming prediction by reflecting a directionality of a correspondingblock using some reference pixels (the first reference pixel), but itmay not accurately reflect the directionality in the block and it maycause the prediction accuracy to decrease. In this case, when using togenerate or correct a prediction block using an additional referencepixel (the second reference pixel), prediction accuracy may beincreased.

To this end, in the example described below, a case in which aprediction block is generated using various reference pixels will bedescribed, and even when terms such as the first and second referencepixels are not used, it may be derived and understood from the abovedefinition.

Whether to support the generation of a prediction block using additionalreference pixels (or prediction pixel or prediction block correction) isdetermined implicitly according to encoding/decoding settings, orinformation on whether it is supported may be explicitly included in aunit such as a sequence, picture, slice, and tile. When prediction pixelcorrection is supported, detailed settings (for example, informationabout applying correction. <when correction is applied> reference pixelinformation used for correction, a weight setting applied to a referencepixel, etc.) related to using additional reference pixels may beimplicitly determined according to encoding/decoding settings, or may beexplicitly determined in a unit of a picture, slice, tile, block, or thelike. Alternatively, the explicit and implicit cases may be mixedaccording to encoding/decoding settings, and the above setting may bedetermined.

In this case, encoding/decoding settings may be defined according to oneor a combination of two or more of an image type, color component, blocksize/shape/position, horizontal/vertical length ratio of block,prediction mode, and (prediction) pixel position.

In the examples described below, the case of implicit processing will bedescribed, but the present invention may not be limited thereto, andexamples of other modifications (explicit case or case of mixed use) maybe possible.

Depending on a prediction mode, prediction blocks can be generated invarious ways. Various cases of correction according to the predictionmode will be described through examples described below. In the exampleto be described later, the case of that a region including the mainreference pixel (or the first reference pixel) is used in a predictionprocess and other reference pixels (or the second reference pixel) areused in the correction process will be mainly described. However, itshould be understood that the case of that the main reference pixel usedin the prediction process (first prediction) is also used in thecorrection process (second prediction) may not be separately mentioned,but may be implemented by being substituted or combined in the exampledescribed below.

In the following, the prediction according to the horizontal mode willbe described.

In the horizontal mode in which the left block is used as a referencepixel (Ref_L in FIG. 9), a prediction block may be generated in ahorizontal direction using adjacent pixels (L0 to L3) of thecorresponding block.

In addition, a prediction block may be generated (or corrected) using areference pixel (Ref_TL, Ref_T, Ref_TR in FIG. 9) adjacent to thecurrent block corresponding to (or parallel to) the predictiondirection. In detail, prediction values may be corrected using adjacentpixels (TL, T0 to T3, and R0 to R3) of the corresponding block.

For example, correction may be performed using one or more secondreference pixels (TL, T1) corresponding to the current pixel (f) and thefirst reference pixel (L1). In detail, it may be a case where correctionis performed by indirectly obtaining a change (f−L1) in a pixel valueaccording to a distance difference (x component difference between f andL1) between a current pixel and a first reference pixel from secondreference pixels (T1 and TL) (for example, T1−TL, etc.).

In this case, the distance difference between the second referencepixels may be some examples (T1 and TL), and various cases {For example,(T1−TL)/(R0−TL), (T1×2+T0+T2)−TL, etc.} in which pixel value changes (orslope information) of two or more pixels are applied may be possible.

In summary, reference pixels involved in generating prediction values ofthe current pixel (f) may include one or more preset pixels (L1, T1, TL)according to the position of the current pixel (x and y components. forexample, x, y component values measured based on specific coordinates<coordinates of the top left block of the current block, such as a inFIG. 9 or TL in FIG. 9, etc.) and the prediction mode. In this case,various encoding/decoding elements as well as the position of thecurrent pixel and the prediction mode may be considered for a weightapplied to each reference pixel, and a portion thereof will be describedlater.

In the case of the vertical mode, a detailed description is omittedbecause a prediction method can be derived by applying only differentdirections to the prediction method of the horizontal mode.

In the following, we will look at prediction according to some diagonalmodes (Diagonal up right). The diagonal mode of this example means adirectional mode that is directed to the top right by using the bottomleft as the starting point of prediction, as shown in the modes 2 to 17modes of FIG. 4, and for convenience of description, it is assumed thatthe mode is the mode 2. In the diagonal mode of this example as well asin the other diagonal modes (Diagonal down left, Diagonal down right), adescription will be given focusing on the case where prediction andcorrection are performed with only pixels of an integer unit.

The diagonal mode is a case in which left and bottom left blocks areused as reference pixels (Ref_L, Ref_BL in FIG. 9), and predictionblocks may be generated in a diagonal direction using adjacent pixels(L0 to L3, B0 to B3) of the corresponding block.

In addition, a prediction block may be generated using reference pixels(Ref_T, Ref_TR in FIG. 9) adjacent to a current block corresponding (orinverted) to a prediction direction. Specifically, a prediction valuemay be corrected using adjacent pixels (T0 to T3, R0 to R3) of thecorresponding block.

For example, correction may be performed using one or more secondreference pixels (T3) corresponding to the current pixel (f) and thefirst reference pixel (L3). Specifically, it may be a case of performingcorrection by estimating a current pixel with a pixel value {forexample, L3×w+T3×(1−w) etc. where w is a weight depending on aninterpolation position} obtained due to linear interpolation between thefirst reference pixel (L3) and the second reference pixel (T3).

In this case, the above example is a case where the second referencepixel used for correction is one, and various cases in which two or moresecond reference pixels are used may be possible. For example, thesecond reference pixel may be corrected {for example, T2*. T2* is(T3×2+T2+R0), etc.} by using additional the second reference pixels (T2,R0) in addition to the second reference pixel (T3) corresponding to thefirst reference pixel, and then used to correct the current pixel.

The use of a plurality of second reference pixels may be understood asperforming filtering (e.g., low pass filter, etc.) with adjacent pixelsbefore correction for the purpose of removing quantization errors andthe like included in the second reference pixels used directly forcorrection. In this case, for the first reference pixel, it is assumedthat a similar operation is performed in a previous intra process (orstep) such as reference pixel filtering, but otherwise, it should beunderstood that one or more pixels may also be used.

In summary, a reference pixel involved in generating a prediction valueof the current pixel (f) may include one or more preset pixels (L3, T3)according to the position of the current pixel and a prediction mode. Inthis case, weights applied to each reference pixel will be describedlater.

In the above-described example, the reference pixel (both the first andsecond reference pixels) used for intra prediction is a case related toa prediction mode of an integer unit pixel. However, according to thedirectionality of the prediction mode (for example, the 3rd mode of FIG.4), not only an integer unit pixel but a decimal unit pixel may be usedfor prediction.

In this example, it is assumed that the first reference pixel isconfigured in a reference pixel memory through the reference pixelinterpolation unit, up to a decimal pixel. If the second reference pixelis also obtainable from the reference pixel memory configured throughthe reference pixel interpolation unit, the second reference pixel mayacquire one or more integer unit pixels or one or more decimal unitpixels at positions corresponding to the first reference pixel.

If, in the reference pixel interpolation unit, a decimal reference pixelcorresponding to a prediction direction is not configured in the memory,an interpolation process of the second reference pixel may be required.

For example, in the case of the prediction mode 3 in FIG. 4, the firstreference pixel used in the current pixel (f) may be a decimal unit andmay have an interpolated pixel value {for example, L2×w+L3×(1−w). wherew is a weight depending on an interpolation position} between two ormore reference pixels (L2 and L3. or L1, L2, L3 and B0, etc.). Thecorresponding second reference pixel may also be a decimal unit, and mayhave an interpolated pixel value between two integer unit referencepixels (T3 and R0).

Since the interpolation of the second reference pixel as in the aboveexample may lead to an increase in the amount of computational quantity,the second reference pixel may be alternately obtained through thefollowing method.

For example, the second reference pixel (between T3 and R0)corresponding to the first reference pixel (between L2 and L3) may bereplaced with one of the adjacent integer unit pixels (T3, R0).Specifically, it may be replaced by an integer unit pixel close to thesecond reference pixel. Alternatively, weights may be assigned to two ormore adjacent integer unit pixels (T3 and R0. or T2, T3, and R0. or T3,R0, and R1. or the like). In detail, a higher weight may be assigned toan integer unit pixel close to the second reference pixel. Since thisexample is a description applicable to other diagonal modes, a detaileddescription is omitted in the example described below.

In some diagonal modes (Diagonal down left), detailed descriptions areomitted because it can be derived by applying only different directionsto the prediction method of the diagonal mode (Diagonal up right) in theabove example.

In the following, we look at the prediction according to some diagonalmodes (Diagonal down right). The diagonal mode of the present examplerefers to a directional mode that directed to the bottom right with thetop left as the starting point of the prediction as in the modes 19 to49 of FIG. 4, and for convenience of explanation, it is assumed that themode is the mode 34.

The diagonal mode is a case where the left, top left, and top blocks areused as reference pixels (Ref_L, Ref_TL, Ref_T in FIG. 9), and aprediction block can be generated in a diagonal direction by usingadjacent pixels (TL, L0 to L3, T0 to T3) of the corresponding block.

In addition, a prediction block may be generated using reference pixels(Ref_BL, Ref_TR in FIG. 9) adjacent to a current block corresponding to(or orthogonal to) a prediction direction. In detail, prediction valuesmay be corrected by using adjacent pixels (B0 to B3, R0 to R3) of thecorresponding block.

For example, correction may be performed using one or more secondreference pixels (B1, R1) corresponding to the current pixel (k) and thefirst reference pixel (TL). In detail, it may be a case of performingcorrection by estimating a current pixel with a pixel value {forexample, B1×w+R1×(1−w) etc. where w is a weight depending on aninterpolation position} obtained due to linear interpolation between thesecond reference pixels (B1 and R1).

In this case, the example is a case where the number of interpolationvalues between the second reference pixels used for correction is one,and various cases in which correction is performed using two or moreinterpolation values may be possible. For example, linear interpolationbetween the reference pixels (e.g., B0 and R1, or B3 and T3) that arenot orthogonal to the prediction direction of the current pixel (k) butcorrespond (or symmetric) to each other of the current pixel can beperformed additionally. That is, the reference pixels that influenceprediction on the current pixel correspond to the first reference pixel(TL) and the second reference pixel (B1 and R1, B0 and R1, B3 and T3,etc.), and some of the second reference pixels (B1 and R1, B0 and R1, B3and T3) may be located in a symmetrical position of the current pixel.

In summary, the reference pixel involved in generating prediction valuesof the current pixel (k) may include one or more preset pixels (B1, R1)according to the current pixel position and the prediction mode. In thiscase, the weight applied to the reference pixel will be described later.

In the following, prediction according to a non-directional mode will bedescribed.

The DC mode is a case that the left and top blocks are used as referencepixels (Ref_L, Ref_T in FIG. 9), a prediction block can be generatedafter acquiring a preset value (e.g., average, etc.) using adjacentpixels of the corresponding block, but is not limited thereto.

In addition, a prediction block may be generated using additionalreference pixels (Ref_TL, Ref_BL, Ref_TR in FIG. 9) adjacent to thecurrent block. In detail, prediction values may be corrected usingadjacent pixels (TL, B0 to B3, R0 to R3) of the corresponding block.

For example, correction may be performed using one or more referencepixels (the first and second reference pixels) corresponding to (oradjacent to) the current pixel. Specifically, the correction may beperformed by using one first reference pixel (T2) closest to somecurrent pixels (g), the correction may be performed by using two or morefirst reference pixels (L1, T1) closest to some current pixels (f), thecorrection may be performed by using two or more first reference pixels(T0, T1, T2) adjacent to some current pixels (b), or the correction maybe performed by using one or more first reference pixels (L0, T0)adjacent to some current pixels (a) and one second reference pixel (TL).

Meanwhile, the Planar mode is a mode in which prediction values aregenerated by linear interpolation through left, right, top, and bottomblocks, and among them, the left and top blocks (Ref_L, Ref_T in FIG. 9)may use adjacent pixels (L0 to L3, T0 to T3) of the corresponding blockas reference pixels. On the other hand, the right and bottom blocks,which are unavailable locations, may be obtained in the existingreference pixel region (Ref_TL, Ref_T, Ref_TR, Ref_L, Ref_BL in FIG. 9)to configure the reference pixel at the corresponding position.

For example, the right and bottom blocks may be filled or derived (forexample, use of methods such as filling the right block by copying onepixel of R0, R1 as it is, or filling the right block with valuesobtained by applying filtering to R0, R1, etc.) with data of someadjacent regions (Ref_TR, Ref_BL) respectively, and may be derived andfilled (for example, interpolation is applied using the data of Ref_TRand Ref_BL) from data of two or more regions. Alternatively, it may befilled with data of some existing regions (Ref_TL, Ref_T, Ref_L) as itis, or it may be derived (for example, the weighted average of one ortwo or more pixels located in Ref_TL, Ref_T, Ref_L is filled in thebottom right block position of the current block, and data andinterpolation of Ref_TR and Ref_BL are applied) from the data andfilled.

Through the above process, the left, right, top and bottom blocks may beused as reference pixels (Ref_T, Ref_L, Ref_TR, Ref_BL in FIG. 9) togenerate a prediction block through an interpolation process. In thiscase, in the interpolation process, the first interpolation may beperformed in the vertical and horizontal directions, and the finalprediction value may be obtained by applying a weighted average to thefirst interpolation value, but is not limited thereto.

In addition, a prediction block may be generated using an additionalreference pixel (Ref_TL in FIG. 9) adjacent to the current block.Specifically, prediction values may be corrected by using adjacentpixels (TL) of the corresponding block.

For example, correction may be performed using one or more referencepixels (the first and second reference pixels) corresponding to thecurrent pixel (f).

In detail, after interpolation in the horizontal direction (using TL andR0) with assuming that the reference pixel (T1) corresponding to thecurrent pixel is a virtual pixel, the obtained interpolation value canbe compared with the corresponding reference pixel (T1) to correct thefirst interpolation value according to the horizontal direction of thecurrent pixel.

In addition, after interpolation in the vertical direction (using TL andB0) with assuming that the reference pixel (L1) corresponding to thecurrent pixel is a virtual pixel, the obtained interpolation value canbe compared with the corresponding reference pixel (L1) to correct thefirst interpolation value according to the vertical direction of thecurrent pixel.

The final prediction value may be obtained using the corrected firstinterpolation value. That is, through the value obtained after assumingthat the reference pixel is a virtual pixel, an effect of correcting thefinal predicted value obtained by interpolation in the horizontal andvertical directions of the current block may be obtained.

In summary, the reference pixel involved in generating the predictionvalue of the current pixel (f) may include one or more preset pixels(TL, T1, L1, R0, B0) according to the current pixel, the predictionmode, etc. In this case, weights applied to each reference pixel will bedescribed later.

In the following, prediction according to a color copy mode will bedescribed.

A color copy mode is a case in which a corresponding block in anothercolor space is used as a reference pixel (Ref_C in FIG. 9), and aprediction block may be generated using pixels in the correspondingblock.

In addition, a prediction block may be generated using adjacentreference pixels (Ref_TL, Ref_T, Ref_TR, Ref_L, Ref_BL) in the currentcolor space. Specifically, the prediction value may be corrected byusing adjacent pixels (T1, T0 to T3, R0 to R3, L0 to L3, B0 to B3) ofthe corresponding block.

In the case of a color copy mode, there is only a difference between theabove-described prediction mode and the prediction method, and aprediction block may be generated (or corrected) using a reference pixelin the same or similar way. It should be understood that variousexamples related to the correction of the other prediction modesdescribed above can be applied or derived in the same or similar mannerto the color copy mode even though the contents are not mentioned in theexamples described below.

For example, correction may be performed using one or more secondreference pixels corresponding to the current pixel. Specifically, thesecond reference pixel adjacent to the current pixel may be used, but areference pixel having the same or similar x component or y component ofthe current pixel may be used for correction. When the current pixel islocated at k, reference pixels such as <T2>, <L2>, <T2, L2>, <T1, T2,T3>, <L1, L2, L3>, <T1, T2, T3, L1, L2, L3> can be used for correction.

Alternatively, a reference pixel (the number is z. z is an integer suchas 1, 2, 3, the number of pixel referred in each direction may be z, orthe number of pixels referred in all directions may be z) positioned inone or more directions of a horizontal, vertical, or diagonal direction(Diagonal down right, Diagonal down left, Diagonal up right, etc.) ofthe current pixel may be used for correction. Specifically, when thecurrent pixel is located at c, reference pixels adjacent to the verticaldirection such as <T2>, <T1, T2, T3> may be used for correction. Whenthe current pixel is located at m, reference pixels adjacent to thehorizontal direction such as <L3>, <L3, B0> may be used for correction.When the current pixel is located at j, reference pixels adjacent in thehorizontal and vertical directions such as <T1, L2>, <T0, T1, T2, L1,L2, L3> may be used for correction. In addition, when the current pixelis located at k, one or more diagonally adjacent reference pixels suchas <R0>, <B0>, <TL>, <R0, B0>, <TL, R0, B0> may be used for correction.

In summary, a reference pixel involved in generating the predictionvalue of the current pixel (f) may include one or more preset pixels(ff, T1, L1, TL) according to the current pixel, a prediction mode, etc.In this case, weights applied to each reference pixel will be describedlater.

Through the above example, a case where prediction is performed usingvarious reference pixels according to a prediction mode has beendescribed. Although the above example has been described on theassumption that correction is applied, whether to correct the currentblock may be determined according to encoding/decoding settings.

In addition, the above example has been described on the assumption thatthere is no restriction on the pixel to be corrected (when correction isapplied), but correction may be performed on all or some pixels in thecurrent block according to encoding/decoding settings.

In this case, when it is limited to some pixels, the correspondingpixels may exist in a single unit, may exist in a line unit such as avertical/horizontal/diagonal line, or may exist in a sub-block unit. Forexample, in FIG. 9, correction targets may be limited to pixels such asa, b, and c. It may be limited to pixels (4×1, 1×3) in a row and columnsuch as a to d or a, e, i. It may be limited to pixels (3×2) in arectangular unit such as a to c and e to g. For convenience ofdescription, a description will be given focusing on a case in which thepixel to which correction is applied is determined in a unit of a linewhen the pixel to be corrected is limited to a part.

In addition, the above example has described a case where the number,position, etc. of reference pixels, which are elements affectingprediction values, are fixed, but the reference pixel setting may bedetermined according to encoding/decoding settings. In addition,although the above example briefly described a weight setting applied toeach reference pixel that affects prediction values, the weight settingfor the reference pixel may be determined according to encoding/decodingsettings.

The above-described encoding/decoding settings may be defined accordingto one or a combination of two or more of an image type, colorcomponent, prediction mode, block size/shape/position, blockwidth/length ratio, pixel position in the block, and the like.

The above description has been mentioned as a case where the detailedsettings regarding intra prediction (or correction) are determinedimplicitly, but the detailed settings may be explicitly determined, andthe detailed settings may include relevant information in a unit such asa sequence, picture, slice, tile, block, etc.

FIG. 10 is a flowchart for explaining an implementation example of animage encoding method according to an embodiment of the presentinvention. In detail, it means a flow chart assuming that the intraprediction (using the first reference pixel) and the correction process(using the first reference pixel and the second reference pixel) areseparated. In this example, it is assumed that a block having an M×Nsize is obtained as the size of the prediction block through theabove-described block division unit.

For the purpose of selecting the optimal prediction mode, one of theprediction mode candidate group is set as a candidate prediction mode(S1010). In general, a configuration of a prediction mode candidategroup may be different according to a color component, and a fixed oneprediction mode candidate group may be set in each color component, butit may not be limited thereto, and various candidate groupconfigurations may be possible.

A prediction block is generated by performing intra prediction accordingto the candidate prediction mode selected through the above process(S1020). An example of prediction block generation has been describedabove, so a description thereof is omitted.

A correction setting for intra prediction is determined according to thesize of the current block and the candidate prediction mode (S1030). Inthis example, it is assumed that the encoding element considered for thecorrection setting is the current block size and the prediction mode,but it has already been mentioned that the correction setting can bedetermined by considering various encoding/decoding elements.

A correction process for the prediction block is performed according tothe determined correction setting (S1040). In this case, the correctionsetting may include not only whether correction is performed, but alsothe location and number of reference pixels, weights of each referencepixel, and pixels to be corrected when correction is performed.

The optimal prediction mode is selected in consideration of the encodingcost of image data for each candidate of the prediction mode (S1050). Inaddition, a bitstream including image data encoded with the determinedoptimal prediction mode and information on the prediction mode isgenerated (S1060).

In this case, the image data may include data on the residual componentof the current block, and information on the size and shape of thecurrent block (e.g., block division information, etc.) may be includedin addition to the prediction mode information. The data related to theresidual component includes scan information of the residual component,whether a non-zero residual coefficient is present in a preset block(current block or subblock of the current block), whether the residualcoefficient is 0, absolute value of the residual coefficient, signinformation, and the like.

FIG. 11 is a flowchart for explaining an example of an implementation ofan image decoding method according to an embodiment of the presentinvention.

The image data extracted from the bitstream and information about theprediction mode are restored (S1110). Then, a prediction block isgenerated by performing intra prediction according to the restoredprediction mode (S1120). A correction setting for intra prediction isdetermined according to the size of the current block and the restoredprediction mode (S1130). The prediction block is corrected according tothe correction setting determined through the above process (S1140). Thecurrent block is reconstructed by adding the reconstructed image dataand the prediction block (S1150). Since the decoding method can derive adetailed description from the encoding method except for someconfigurations (for example, selecting an optimal prediction mode,etc.), the example described below will focus on the encoding method.

Following is a description of detailed settings for intra prediction.Various encoding/decoding elements may be considered as to whether toperform correction of the prediction block. The following is a detailedexample of elements affecting whether or not to perform calibration.

As an example (1), whether to perform correction may be determinedaccording to the size of the block. The small size of the block may be acase in which block division is performed because a region adjacent tothe corresponding block is difficult to predict as one block. Therefore,since the image characteristics between the current block and theadjacent region are likely to be different, additional correction inaddition to prediction using a reference pixel composed of adjacentpixels may adversely affect prediction accuracy. Of course, the abovesituation may not always occur, and vice versa. In addition, the abovecase may not be considered in a situation where it is explicitlydetermined whether to perform the correction. However, in the exampledescribed later including this example, the description will be made inconsideration of the fact that the correction setting is implicitlydescribed.

Correction may be performed when the size of the block is equal to orgreater than/greater than a certain size (M×N. for example, 2^(m)×2^(n),where m and n are 1 or more integers such as 2 to 6). In this case, oneor more boundary values for the size of the block may exist.

For example, correction may not be performed in a block smaller than16×16, and correction may be performed in a block larger than 16×16.Configurations in which explicit cases are mixed here may also bepossible.

For example, correction may not be performed implicitly in a block of8×8 or less, correction may be performed implicitly in a block of 64×64or more, and it is possible to explicitly determine whether to performcorrection in a block greater than 8×8 to less than 64×64.

As an example (2), whether to perform the correction may be determinedaccording to the shape of the block. In the present invention, a blockof a rectangular shape (e.g., 64×8, etc.) may be obtained through ablock division of a binary or ternary tree. In this case, the adjacentblock also has a high probability that the horizontal/vertical lengthratio is similar to the corresponding block, and performing correctionin addition to the prediction using pixels within the adjacent block mayalso negatively affect the accuracy of prediction.

Correction can be performed when the block shape is rectangular and ifthe horizontal/vertical length ratio is equal to or greater than/greaterthan a certain ratio (k:1 or 1:k. k is an integer of 1 or more such as2, 3, 4, etc.), on the other hand, if the ratio is equal to or lessthan/less than the certain ratio, correction cannot be performed.Herein, there may be one or more boundary values for thehorizontal/vertical length ratio of the block.

For example, correction is not performed in a block having ahorizontal/vertical length ratio of less than 4:1 or 1:4, such as 16×16or 32×64. In a block such as 8×32 or 64×16, the horizontal/verticallength ratio is greater than 4:1 or 1:4, correction can be performed.Configurations in which explicit cases are mixed here may also bepossible.

For example, in a block having a square shape (i.e., 1:1) such as 16×16,the implicit correction is performed. In a block having ahorizontal/vertical length ratio of 4:1 or 1:4, such as 64×8, nocorrection is performed implicitly. In a block having a rectangularshape and the ratio of less than 4:1 or 1:4, whether to perform thecorrection can be decided explicitly.

As an example (3), whether to perform the correction may be determinedaccording to the prediction mode. Applying correction may be aimed atimproving the accuracy of the prediction. Therefore, it is intended toutilize a directional characteristic of a prediction block that has notbeen supplemented in the existing prediction process, and someprediction modes may have a prediction mode (or method) that isdifficult to utilize. This may also be a different story in the explicitcase, but this example will also focus on the implied case. Theprediction mode in which correction is performed (or allowed) may beconfigured in various combinations.

For example, correction may be performed on all or part of thedirectional mode, and vertical, horizontal, and diagonal modes (modes 2,34, and 66 in FIG. 4), etc. may be included in correctable modes.Specifically, configurations such as <Vertical+Horizontal>,<Vertical+Horizontal+34>, <Vertical+Horizontal+2+66>,<Vertical+Horizontal+2+34+66>, etc. can be possible.

In addition, it is possible to perform correction on a prediction modewithin a certain predetermined range (for example, in the mode a, m isincluded on the left and n is included on the right. in this case, m andn are integers of 1 or more such as 1, 2, 3, etc.) based on someprediction modes (vertical, horizontal, diagonal mode, etc.). As adetailed example, in the vertical mode (50), correction may be performedin the 48, 49, 51, and 52 modes (m and n are 2).

Alternatively, correction may be performed on all or part of thenon-directional mode, and correction may be performed on all or part ofthe color copy mode.

In summary, prediction modes to be corrected can be classified invarious combinations (some directional modes+non-directional mode, somedirectional modes+color copy mode, some directionalmodes+non-directional mode+some color copy modes, etc.) in theprediction mode candidate group, and correction can be performed whenthe prediction mode is one of the classified modes. In addition, aconfiguration in which explicit cases are mixed may also be possible.

For example, correction cannot be performed in a block having someprediction modes (modes with similar directionality, including Diagonaldown right in this example). The correction can be performed in a blockhaving some prediction modes (modes with similar directionality,including a non-directional mode, color copy mode, horizontal andvertical mode in this example). In some prediction modes (modes withsimilar directionality including Diagonal down left and Diagonal upright in this example), it is possible to explicitly determine whetherto perform correction.

In the case of the color copy mode, it has been described that one ormore correlation information can be obtained, which means that aplurality of color copy modes are supported. That is, it means that aplurality of color copy modes can be supported due to differences indetailed settings for obtaining correlation information. In this case,whether to perform the correction may be determined according to thesetting of the color copy mode (that is, how to obtain correlationinformation, etc.).

FIG. 12 is an exemplary diagram for detailed setting of a color copymode according to an embodiment of the present invention. In detail, ato c of FIG. 12 may be classified according to a region (hatched portionin the drawing) referenced (or compared) to obtain correlationinformation. In this example, it is assumed on the assumption that thesettings related to obtaining the other correlation information are thesame and the settings of the regions to be compared are different forobtaining the correlation information, but the color copy mode may bedivided into other detailed settings, and it should also be understoodwhether to perform the correction can be determined accordingly.

The a to c in FIG. 12 are set as regions where left and top blocks, topand top right blocks, left and left bottom blocks of a blockcorresponding to the current block are compared with each other. Inother words, it means that the correlation information is obtained fromthe adjacent block, and the data obtained by applying the correlationinformation to the block corresponding to the current block will be usedfor the prediction block of the current block.

Referring to a of FIG. 12, it may mean that a current block and a regionthat binds the left and top blocks adjacent to it and a correspondingblock of a different color space and a region that binds the left andtop blocks adjacent to it have a high correlation with each other. Inthis example, it is assumed that the reference pixel used for thecorrection of the current block is limited to the left and top blocks ofthe current block.

Therefore, the a in FIG. 12, it can be expected that when acquiring aprediction block in another color space and performing correction, usingadjacent blocks, left and top blocks, may help improve predictionaccuracy. In particular, an effect of reducing deterioration occurringat the boundary between blocks may also occur.

Referring to b and c of FIG. 12, it may mean that a current block and aregion that binds the top and top right blocks adjacent to it and acorresponding block of a different color space and a region that bindsthe top and top right blocks adjacent to it have a high correlation witheach other. In this case, performing correction using the left and topblocks can be rather negative for the accuracy of the prediction.Particularly, when b to c are selected in the case where all of a, b,and c in FIG. 12 are supported (three modes), the adverse effect may bemuch greater.

In this case, it may be more efficient to perform correction using thetop and top right blocks, but in this example, the reference pixel usedfor correction may be limited to the left and top blocks, so it can beefficient to have a setting that implicitly determines not to performthe correction.

In summary, correction may be performed on all or part of the color copymode, which may be determined based on setting information regarding thecolor copy mode. In this case, since the setting information can derivea related description in the above-described color copying mode,detailed description is omitted.

The above examples (No. 1 to 3) indicate a case in which correction isperformed by a single encoding/decoding element, and examples of variousmodifications including the opposite case of the above examples may bepossible. In addition, it may be possible to determine whethercorrection is performed by combining two or more elements, and otherencoding/decoding elements not mentioned in examples 1 to 3 may also beconsidered as included in the encoding/decoding setting for determiningwhether to perform the correction.

Following is a description of detailed settings for intra prediction.When correction of the prediction block is performed, all or some pixelsin the block may be considered as correction targets. For convenience ofdescription, it is assumed that only some pixels are considered ascorrection targets, and a detailed example of elements affecting thedetermination of the correction target pixels will be described below.

FIG. 13 is a conceptual diagram illustrating various configurationexamples of pixels to be corrected in a current block according to anembodiment of the present invention.

Referring to FIG. 13, pixels in a block may be represented by Z_(x,y),and a row and column in a block may be represented by R_(k) and C_(k),respectively. It is assumed that when a pixel to be corrected isexpressed individually, it is expressed as Z, and when expressed as aline unit, it is expressed as R and C.

As an example (1), the pixel to be corrected may be determined accordingto the size of the block. In the case of a block such as 64×64, themaximum distance between the reference pixel and the pixel in the blockmay be 63 (based on horizontal/vertical), whereas in the case of a blocksuch as 8×8, it may be 7. It can be seen from the above example that itis necessary to determine the pixel to be corrected in consideration ofthe size of the block. Of course, again, it is necessary to set asideand understand the explicit case.

The pixel and position to be corrected may be determined based on thesize of the block (2^(M)×2^(N)), and a setting proportional to the sizeof the block may be set.

For example, a certain region (For example, 2^(M)×2^(S), 2^(T)×2^(N). Inthis case, the T and S are M and N minus v, w. The v and w are integerssuch as 0, 1, 2, etc.) obtained in proportion to the size of the block(for example, the range of pixels to be corrected may be proportional tothe size of the current block, or may be proportional to an exponent of2 when expressed as an exponent of 2) may be determined as a pixel to becorrected. Specifically, when the current block is 4×4 as shown in FIG.13, the pixel to be corrected may be seven pixels represented by R₀ andC₀ (4×1, 1×4. when set based on the top left of the block). In thiscase, there may be one or more boundary values related to the size ofthe block, which may affect the determination of the pixel to becorrected.

For example, when the size of the block is equal to or less than/lessthan a certain size (A×B. for example, 2^(a)×2^(b), where a and b areintegers greater than or equal to 1 such as 2 to 6), a certain regionmay be determined as a pixel to be corrected in proportion to the sizeof the block, and when the size is equal to or greater than/greater thanthe certain size, a preset region may be determined as a pixel to becorrected.

As a further example, when the size of the block is 4×4 or less, thepixel to be corrected (for example, 12 pixels represented by R₀, R₁, andC₀, C₁ when the size is 4×4) may be determined according to the size ofthe block, and when the size is 4×4 or more, a pixel to be corrected maybe determined as a preset region (12 pixels represented by R₀, R1, andC₀, C₁ in the same manner as the pixel to be corrected when the boundaryvalue is applied). In this case, a configuration in which an explicitcase is mixed may also be possible. A description of this may lead to arelated description in various examples described above.

As an example (2), a pixel to be corrected may be determined accordingto a block shape. For a block such as 128×16, consider the referencepixel and the pixel in the block based on the bottom right pixel. Inthis case, the maximum distance between the reference pixel and thebottom right pixel may be 15 in the vertical direction and 127 in thehorizontal direction. The above example may also be a case indicatingthat it is necessary to determine the pixel to be corrected according tothe shape of the block.

Pixels and positions to be corrected may be determined based on theshape of the block (or the horizontal/vertical length ratio of theblock. 2^(M):2^(N) or M:N when the current block is 2^(M)×2^(N)), and asetting proportional to the shape of the block may be set.

For example, a certain region (M×N or 2^(S)×2^(T). In this case, the Sand T are M and N <where M and N are exponential values of blocklengths> minus v, w. The v and w are integers of 0, −1, 1, etc. Thisexample is described as M×N) obtained according to thehorizontal/vertical length ratio of a block (compared to M:N in thisexample) may be determined as a pixel to be corrected. Specifically,when the current block is 8×4, the pixel to be corrected may be 26pixels represented by R0 to R2 (3 rows) and C₀ to C₁ (2 columns),although not illustrated in FIG. 13. In this case, there may be one ormore boundary values related to the horizontal/vertical length ratio ofthe block, which may affect the determination of the pixel to becorrected.

For example, when the horizontal/vertical length ratio of a block isequal to or less than/less than a certain ratio (k:1 or 1:k. k is aninteger greater than or equal to 2, 3, 4, etc.), a certain region may bedetermined as a pixel to be corrected in proportion to thehorizontal/vertical length ratio of the block, and when the ratio isequal to or greater than/greater than the certain ratio, a preset regionmay be determined as a pixel to be corrected.

As an additional example, when the horizontal/vertical length ratio ofthe block is 2:1 (or 1:2) or less, the pixel to be corrected (set thesame as in the above example, i.e., 2 rows and 1 column or 1 row and 2columns) may be determined according to the horizontal/vertical lengthratio of the block, and when the ratio is greater than 2:1 (or 1:2), apixel to be corrected may be determined as a preset region (when itcorresponds to the boundary value, it applies the same as the pixelsetting to be corrected). In this case, a configuration in which anexplicit case is mixed may be possible, and a related description may bederived from the above-described example.

As an example (3), a pixel to be corrected may be determined accordingto a prediction mode. Depending on the prediction mode (or method), thedistance between the reference pixel and the pixel in the block may beconsidered. When considering the top right pixel reference of the block,the top pixel is the reference pixel, but in the case of a directionalmode such as diagonal up right, there may be many distance differencesfrom the pixel actually referred to in prediction. As described above,it may be necessary to consider the pixel to be corrected according tothe prediction mode.

A pixel to be corrected may be determined based on detailed settings ofthe prediction mode (prediction method, directionality, reference pixelposition, etc.). The following describes the case of each predictionmode by classifying the directional mode, the non-directional mode, andthe color copy mode. In the example to be described later, a drawing forindicating a pixel in a block and a reference pixel refers to FIG. 9,and a drawing for indicating a pixel to be corrected refers to FIG. 13.Even if the drawing numbers are not mentioned, the size of the exampleblocks of the two drawings is the same and the drawing symbols are notoverlapped, so detailed examples can be derived through correspondingdrawing symbols in each drawing.

In the directional mode (used for prediction or correction), a pixel tobe corrected may be determined based on the position of the referencepixel, the distance between the pixel to be corrected and the referencepixel, and the direction (or angle) of the prediction mode.

In some prediction modes (diagonal down left), the first reference pixelfor prediction of the pixel m may be R0. Since the distance from thefirst reference pixel used for prediction is far, correction may beperformed through the second reference pixel (B0 in this example)corresponding to the prediction direction. Meanwhile, in the case of thepixel d, since the distance from the second reference pixel (B0 in thisexample) corresponding to the prediction direction is far, correctionmay not be performed. That is, a region adjacent to the first referencepixel may be excluded from the correction target, and a region notadjacent to the first reference pixel may be included in the correctiontarget.

For example, in the case of a directional mode such as diagonal downleft, reference pixels corresponding to C₀ and C₁ (or R₂ and R₃) belongto the correction target, and other pixels may be excluded from thecorrection target. A directional mode such as diagonal up right can bederived by changing the direction and the prediction mode, so detaileddescription is omitted.

In some prediction modes (diagonal downright), the first reference pixelfor prediction of the pixel a may be TL. Since the distance between thesecond reference pixels (T1, L1 in this example) corresponding to theprediction direction is close, correction may be performed through this.Meanwhile, in the case of the pixel p, since the distance from thesecond reference pixel corresponding to the prediction direction is alsofar, correction may not be performed. That is, a region adjacent to thefirst reference pixel and adjacent to the second reference pixel may beincluded in the correction target, and may be excluded from thecorrection target in other cases.

For example, in a directional mode such as diagonal down right,reference pixels corresponding to C₀, C₁, R₀, R₁ belong to thecorrection target, and other pixels may be excluded from the correctiontarget.

In some prediction modes (horizontal mode), the first reference pixelfor prediction of the pixel d may be L0. When comparing L0 with pixels ato c used as the first reference pixel, the distance from the firstreference pixel may be far, but the distance from the second referencepixel (T3, TL in this example, but for comparison of the degree ofchange, TL is compared to L0 and T3 can be compared to the pixel d, sothe distance from T3 is assumed) corresponding to the predictiondirection may be close. That is, a region adjacent to the secondreference pixel, regardless of whether it is adjacent to the firstreference pixel, may be included in the correction target, and may beexcluded from the correction target in other cases.

For example, in a directional mode such as a horizontal mode, referencepixels corresponding to R₀, R₁ may be included in the correction target,and in other cases, the reference pixel may be excluded from thecorrection target. Since a directional mode such as a vertical mode canbe derived by changing the prediction mode and the direction, a detaileddescription is omitted.

In the non-directional mode, a pixel to be corrected may be determinedbased on the position of the reference pixel, the distance between thetarget pixel and the reference pixel, and the prediction method.

In some prediction modes (DC mode), a method of predicting pixels a to pwith a value derived from a plurality of adjacent pixels instead of asingle reference pixel may be used. In the case of correction, in whicha single reference pixel is used, correction may be performed onadjacent pixels based on the distance between the target pixel and thesecond reference pixel. That is, the region adjacent to the secondreference pixel may be included in the correction target, and otherregions may be excluded from the correction target.

For example, in the case of a non-directional mode such as a DC mode,reference pixels corresponding to R₀ and C₀ may be included in thecorrection target, and in other cases, the reference pixel may beexcluded from the correction target.

In some prediction modes (Planar mode), the first reference pixel may beclassified into a plurality of categories. In some categories a (leftand top blocks), it may be a region that includes data that can be usedin the actual corresponding location. In addition, in some categories b(right and bottom blocks), there is no data available at thecorresponding location, so it may be a region including data derived invarious ways. In this case, correction may be performed on a pixeladjacent to a region including category 1. That is, a region adjacent tothe first reference pixel and a region adjacent to a category classifiedas the category a may be included in the correction target, and otherregions may be excluded from the correction target.

For example, in the case of a non-directional mode such as a planarmode, reference pixels (Z_(0,0), Z_(1,0), Z_(0,1), Z_(1,1)) in which R₀,R₁ and C₀, C₁ overlap may be included in the correction target, and inother cases, it can be excluded from correction. As another example,reference pixels corresponding to R₀, R₁ and C₀, C₁ may be included inthe correction target, and may be excluded from the correction target inother cases. In this case, the overlapping reference pixels (Z_(0,0),Z_(1,0), Z_(0,1), Z_(1,1)) may be used for correction since the pixelscorresponding to the left and top directions of each pixel correspond tothe second reference pixel, and the non-overlapping reference pixels maybe used for correction since the pixels corresponding to the left(Z_(0,2)) or top (Z_(2,0)) direction of each pixel correspond to thesecond reference pixel.

In the example of the directional/non-directional mode, there may be oneor more boundary values (for example, if the x/y component is equal toor greater than/greater than (or equal to or less than/less than) p/qbased on a certain coordinate <assuming that it is the top leftcoordinate here>, it is included in the correction target) based on xand y components to distinguish pixels to be corrected. In addition,various boundary values may be set to determine pixels to be corrected.

The plurality of examples are limited to some examples for selecting atarget to be corrected, and examples of various modifications may bepossible, such as a position in which a pixel to be corrected is locatedin an opposite position in a block or in a vertical/horizontal symmetricposition. In addition, the above example may be applied to the same orsimilar application to the color copying mode described later.

In the case of the color copy mode, a pixel to be corrected may bedetermined based on the position of the reference pixel, the distancebetween the pixel and the reference pixel, and some settings (Forexample, color copy mode selection information, correlation informationacquisition location, etc.) of the color copy mode.

In the following, it is assumed that correlation information is obtainedby referring to the left and top blocks as shown in the a of FIG. 12. Ifthe regions to be compared for obtaining correlation information aredifferent as shown in b and c of FIG. 12, a related description can bederived from a of FIG. 12.

In the color copy mode, the pixel d may be corrected through one or moresecond reference pixels (T3 in this example, or T2, T3, R0)corresponding to the vertical direction. Meanwhile, the 1 pixel maycorrespond to the second reference pixel which is the same or similar tothe pixel d in the vertical direction, but correction may not beperformed because the distance from the second reference pixel is far.That is, a region adjacent to the second reference pixel in the verticaldirection may be included in the correction target, and a region notadjacent to the second reference pixel may be excluded from thecorrection target.

For example, reference pixels corresponding to R₀, R₁ belong to thecorrection target, and pixels belonging to R₂, R₃ may be excluded fromthe correction target. In this example, the boundary value (w in R_(w).,or y in Z_(x,y). in this case, a boundary value such as w or y is avalue equal to or more than/more than 0˜equal to or less than/less thanthe vertical length of a block) for classifying the correction targetmay be 2.

In the color copy mode, the m pixel may be corrected through one or moresecond reference pixels (L3 in this example. Or L2, L3, B0)corresponding to the horizontal direction. On the other hand, the opixel may correspond to the second reference pixel which is the same orsimilar to the pixel m in the horizontal direction, but correction maynot be performed because the distance from the second reference pixel isfar. That is, a region adjacent to the second reference pixel in thehorizontal direction may be included in the correction target, and aregion not adjacent to the second reference pixel may be excluded fromthe correction target.

For example, reference pixels corresponding to C₀, C₁ belong to acorrection target, and pixels belonging to C₂, C₃ may be excluded from acorrection target. In this example, the boundary value (w in C_(w), or xin Z_(x,y). in this case, a boundary value such as w or x is a valueequal to or more than/more than 0˜equal to or less than/less than thehorizontal length of a block) for classifying the correction target maybe 2.

In the color copy mode, pixels f, h, n, and p may be corrected throughrespective second reference pixels corresponding to vertical andhorizontal directions. In the case of the pixel f, since the distancefrom the second reference pixels (T1 and L1 in this example) in thevertical and horizontal directions is close, correction may be performedthrough the corresponding second reference pixels in the vertical andhorizontal directions. In the case of the pixel h, the distance from thecorresponding second reference pixel (T3 in this example) in thevertical direction is close, but since the distance from thecorresponding second reference pixel (L1 in this example) in thehorizontal direction is far, correction may be performed throughcorresponding second reference pixel in the vertical direction. In thecase of the pixel n, the distance from the second reference pixel (L3 inthis example) corresponding to the horizontal direction is close, butsince the distance from the second reference pixel (T1 in this example)corresponding to the vertical direction is far, the correction may beperformed through the second reference pixel corresponding to thehorizontal direction. In the case of the pixel p, since the distancefrom the second reference pixel is far in the vertical and horizontaldirections, correction may not be performed.

For example, reference pixels corresponding to C₀, C₁ and R₀, R₁ belongto the correction target, and other pixels may be excluded from thecorrection target. In this example, the boundary value (x in Z_(x,y), y.in this case, the boundary value of x is a value equal to or morethan/more than 0˜equal to or less than/less than the horizontal lengthof a block, and the boundary value of y is a value equal to or morethan/more than 0˜equal to or less than/less than the vertical length ofa block) for classifying the correction target may be 2 in thehorizontal and vertical directions, respectively.

In this case, the region to be corrected may be classified into a part(Z_(0,0), Z_(0,1), Z_(1,0), Z_(1,1)) that is applied in the horizontaland vertical directions, a part (Z_(2,0), Z_(2,1), Z_(3,0), Z_(3,1))applied only in the horizontal direction, and a part (Z_(0,2), Z_(0,3),Z_(1,2), Z_(1,3)) applied only in the vertical direction. That is, itmay mean that all of the regions belong to a correction target, but thesecond reference pixel used for correction can be classified.Alternatively, only a part commonly applied in the horizontal andvertical directions may be limited to a correction target, and variousboundary values may be set.

FIG. 14 is an exemplary diagram for explaining a case where correctionis performed in a color copy mode according to an embodiment of thepresent invention.

Referring to FIG. 14, an example of a case in which a boundary value forclassifying a correction target is 1 in the horizontal and verticaldirections is shown. In this case, when correction is applied only inthe horizontal or vertical directions, it means a and c in FIG. 14, andwhen correction is commonly applied in the horizontal and verticaldirections, it means b in FIG. 14. This indicates that the secondreference pixel used for correction may be configured differently asdescribed above.

In the color copy mode, the pixel f may be corrected throughcorresponding second reference pixels (T1, L1, TL, T3, L3) in one ormore of vertical, horizontal, and diagonal directions. Meanwhile, in thecase of the pixel p, since the distance from the corresponding secondreference pixel (T3, L3, TL, R3, B3 in each example) is far, correctionmay not be performed.

For example, reference pixels corresponding to Z_(0,0) to Z_(3,0),Z_(0,1) to Z_(2,1), Z_(0,2), Z_(1,2), Z_(0,3) belong to the correctiontarget, and other pixels are excluded from the correction target. Inthis example, the boundary value (Z_(x,y) in w. w may be a value that iscompared by the sum of x and y. w is a value equal to or more than/morethan 0˜equal to or less than/less than sum of the horizontal andvertical lengths) for classifying the correction target may be 3. Inthis case, although the example in which the region to be corrected isdivided by the sum of the horizontal and vertical lengths was given,various other boundary values may be set.

In the case of the color copying mode, a prediction value is obtained ina color space other than the adjacent region of the current block,unlike the existing prediction mode. In general, a region obtainable (orreferenced) based on a block scan order based on a current pixel may belimited to left, top, top left, top right, and bottom left directions ofa target pixel.

However, in the case of the color copy mode, since all the predictionpixels of the current block are obtained in corresponding blocks ofdifferent color spaces, it may also be possible to use them in thecorrection process. In this case, data for prediction may be acquired inthe left, right, top, bottom, top left, top right, bottom left, andbottom right directions of the current pixel.

FIG. 15 is an exemplary diagram for explaining a case where correctionis performed in a color copy mode according to an embodiment of thepresent invention.

Referring to FIG. 15, it shows a case in which a pixel predicted from acorresponding block of a different color space is used as well as anadjacent pixel of the current block for correction of the current block.That is, the corresponding block means a case that is used for intraprediction as the second reference pixel as well as the first referencepixel. The a to c in FIG. 15 illustrate cases in which correction isperformed on pixels adjacent to the top, top left, and left blocks ofthe current block, and will be described mainly with reference to the ain FIG. 15.

The a in FIG. 15 illustrates a case in which correction is performedusing pixels adjacent in the directions of left, right, top, bottom, topleft, top right, bottom left, bottom right, etc. of the pixel c as thesecond reference pixel. Among them, T1, T2, T3 may be pixels adjacent tothe current block, and b, d, f, g, h may be pixels existing (or derived)in corresponding blocks of different color spaces. As shown in thefigure, various second reference pixel configurations may be possible aswell as setting pixels adjacent to all directions as a second referencepixel of a pixel to be corrected.

For example, a second reference pixel configuration used for variouscorrections such as a vertical direction (T2, g), a horizontal direction(b, d), a cross shape (T2, b, d, g), and an X shape (T1, T3, f, h) maybe possible centered on a target pixel. In this case, the secondreference pixel configuration may be determined in consideration ofvarious encoding/decoding elements, such as the position in the block ofthe pixel to be corrected.

For correction, a weight applied to a pixel adjacent to the target pixelmay be variously set, and a weight applied to the target pixel may be aand a weight applied to the adjacent pixel may be b (when total weightsum is 1, <1−a> divided by the number of adjacent pixels). In this case,the weight applied to adjacent pixels can be set to be the same or not.In addition, the weight applied to a may have a positive value, and theweight applied to b may have a positive or negative value.

In the color copy mode of this example, when pixels of the correspondingblock as well as the adjacent region of the current block are used forcorrection, the same or similar configuration may be applied to theexamples described below, and detailed description thereof will beomitted.

In the above example, the region for obtaining the correlationinformation in the color copy mode may correspond to the left and topblocks of the current block. Although the case in which the region ofthe second reference pixel used for correction is not limited has beendescribed through the above example, an example limited to the left andtop blocks of the current block may also be possible. An example of acase in which whether to correct is determined based on the locationwhere the correlation information is obtained is already describedthrough FIG. 12. In this example, it is assumed that correction isperformed, but the description of the case in which the pixel to becorrected can be determined adaptively is continued.

Referring to b (or c) of FIG. 12, it shows an example of a color copymode for obtaining correlation information by referring to the top, topright (or left, bottom left) blocks of the current block, and the topand top right (or left, bottom left) blocks of the corresponding block.

In this case, a pixel to be corrected may be determined based on anadjacent region used for obtaining correlation information.

For example, the b in FIG. 12, regions (For example, R₀, R₁, C₃, C₂,etc.) adjacent to the top and top right blocks of the current block maybe included as pixels to be corrected, and other regions may not beincluded as pixels to be corrected. Meanwhile, the c in FIG. 12, regions(For example, C₀, C₁, R₃, R₂, etc.) adjacent to the left and bottom leftblocks of the current block may be included as pixels to be corrected,and other regions may not be included as pixels to be corrected.

In b of FIG. 12, as a boundary value (when R₀, R₁ are selected) forselecting a correction target, w in R_(W) may be set to be less than 2,or as a boundary value (when C₃, C₂ are selected) for selecting acorrection target, w in C_(W) may be set to 2 or more. The c in FIG. 12may be derived from the previous example, and thus detailed descriptionis omitted. In addition, the boundary value setting mentioned throughthe above-described various embodiments as well as the above example maybe applied to the same or similar in this example.

In the case of the above example, the case in which a pixel to becorrected is selected according to the position of the correlationinformation acquisition region, which is one of the detailed settings ofthe color copy mode, is described, but various other cases in whichpixels to be corrected are selected according to detailed settings ofother color copy modes may be possible.

In addition, the plurality of examples are limited to some examples forselecting a target to be corrected, and examples of variousmodifications may be possible, such as a position in which a pixel to becorrected is located in an opposite position in a block or in avertical/horizontal symmetric position. In addition, the above examplemay be applied to the same or similar application to the above-describedexisting prediction mode.

Through the above-described plurality of examples, a case in which theencoding/decoding setting for selecting a pixel to be corrected isdetermined according to the size, shape, prediction mode, etc. of ablock, and various other elements are considered to determine theencoding/decoding setting. In addition, the encoding/decoding settingmay be defined according to a combination of two or more elements inaddition to being defined according to a single encoding/decodingelement as in the above example. In such a case, the boundary valuesetting for selecting the pixel to be corrected, which has beendescribed through the above example, may also be determined inconsideration of a plurality of encoding/decoding elements. That is, itmeans that a final boundary value setting for selecting a pixel to becorrected may be determined according to encoding/decoding settingsdefined according to one or more encoding/decoding elements.

In the following, a relationship setting between the prediction pixelvalue (or first prediction pixel value) generated through the firstreference pixel and the prediction correction value (or secondprediction pixel value or final prediction pixel value) generatedthrough the first reference pixel or the second reference pixel will bedescribed. In this case, the relationship setting may mean weightsetting applied to each pixel.

The following is an example of an equation in which intra prediction isperformed using one or more reference pixels.

pred_sample′(i,j)=pred_sample(i,j)×W _(p)+ref_sample_a(a _(i) ,a _(j))×W_(a)+ref_sample_b(b _(i) ,b _(j))×W _(b)+ref_sample_c(c _(i) ,c _(j))×W_(c)  [Equation 3]

In the above equation, pred_sample(i, j) and pred_sample*(i, j) mean aprediction pixel value (or first prediction pixel value) and aprediction correction value (or final prediction pixel value) at (i, j),and ref_sampel_a to ref_sample_c refer to reference pixels used in theprediction pixel correction process. In this case, (a_(i),a_(j)),(b_(i),b_(j)), (c_(i),c_(j)) indicate positions of each reference pixelused in the correction process. In addition, w_(p), w_(a), w_(b), w_(c)mean weights applied to each pixel in the process of correcting theprediction pixels.

The number of pixels used (or referenced) in the prediction pixelcorrection process is k (k is an integer of 0 or more such as 1, 2, 3,etc.), which may be fixed or adaptive depending on encoding/decodingsettings. Although the number of reference pixels used in the correctionprocess may be determined according to various encoding/decodingelements, it is assumed that up to three pixels are used in thisexample. When less than three reference pixels are used, it is assumedthat some reference pixels are filled with zeros or weights of somereference pixels are filled with zeros.

When performing correction on a prediction block, at least one referencepixel (except when the actual reference pixel value is 0) that is not 0in the above equation is used for correction, and the weight of thereference pixel may not be 0, and when not performing correction, allthe reference pixels may be filled with 0 in the above equation, or theweights may be set to 0 for all reference pixels. For example, thereference pixel position used for correction may be obtained accordingto the prediction mode, but when it is determined that the correction isnot to be performed by other encoding/decoding elements (e.g., the sizeof the block), it means that the process of setting the weight appliedto the reference pixel to 0 is possible. That is, it can be understoodthat the prediction pixel value (first prediction pixel value) isdetermined as the prediction correction value (second prediction pixelvalue or final prediction pixel value).

(when the correction is performed on the prediction block) for a pixelincluded in the range to be corrected, in the above equation, at leastone reference pixel that is not 0 may be used for correction, and theweight of the reference pixel may not be 0. For a pixel not included inthe range to be corrected, in the above equation, all the referencepixels may be filled with zeros, or the weights of all the referencepixels may be set to zeros.

Although this example performs correction on the prediction block andexplains on the assumption that all pixels in the block are included inthe correction target, it should be understood that a case in which somepixels are included in the correction target can be applied incombination with the contents described below.

A reference pixel used for correction may be variously configuredaccording to a prediction mode. It is assumed that (i,j), (a_(i),a_(j)),(b_(i),b_(j)), (c_(i),c_(j)) are measured based on the top leftcoordinates (0,0) of a prediction block (M×N). The following showsvarious cases in which (a_(i),a_(j)), (b_(i),b_(j)), (c_(i),c_(j)) areset according to a prediction mode.

For example, when a prediction mode is the vertical mode, (−1, −1), (−1,j), (i, −1), or (−1, −1), (−1, j), (−1, N−1) may be set. Alternatively,when the prediction mode is the horizontal mode, (−1, −1), (i, −1), (−1,j), or (−1, −1), (i, −1), (M−1, −1) may be set.

Alternatively, when the prediction mode is the Diagonal up right mode,(−1, −1), (i+j+1, −1), (−1, i+j+1), or (i+j, −1), (i+j+1,−1), (i+j+2,−1) may be set. Alternatively, when the prediction mode is the Diagonaldown left mode, (−1, −1), (−1, i+j+1), (i+j+1, −1), or (−1, i+j), (−1,i+j+1), (−1, i+j+2) may be set.

Alternatively, when the prediction mode is the Diagonal down right mode,(−1, −1), (i+j+1, −1), (−1, i+j+1), or (i−j−1, −1), (i+j+1, −1), (−1,i+j+1)<if i is greater than or equal to j>, or (−1, j−i−1), (−1, i+j+1),(i+j+1, −1)<if i is less than j> may be set.

Alternatively, when the prediction mode is the DC mode, (−1, −1), (−1,j), (i, −1), or (−1, −1), (M, −1), (−1, N) may be set. Alternatively,when the prediction mode is the Planar mode, (−1, −1), (−1, j), (i, −1),or (−1, −1), (M, −1), (−1, N) may be set.

Alternatively, when the prediction mode is the color copy mode, (−1,−1), (i, −1), (−1, j), or (i−1, j), (i, j), (i+1, j), or (i, j−1), (i,j), (i, j+1), or (i−1, j−1), (i, j), (i+1, j+1), or (i−1, j+1), (i, j),(i+1, j−1) may be set.

The above example assumes a reference pixel position determinedaccording to some prediction modes, and in the case of a mode (forexample, the directional mode. assuming the mode 50 in FIG. 4, theadjacent the modes 49, 51, etc.) adjacent to the prediction mode, areference pixel (for example, <i+1, −1>, <i−1, −1>, <i+2, −1>, etc.)adjacent to the reference pixel position (for example, assume <i, −1>)determined according to the prediction mode may be used in thecorrection process.

In addition, various reference pixel configurations may be possible,including the above example. In this case, even if the reference pixelas in the above example is configured according to each prediction mode,it may not mean that a weight other than 0 is always applied to thecorresponding reference pixel (or all reference pixels).

The weight applied to the reference pixel used in the correction processmay be set based on the horizontal (M) and vertical (N) lengths (orexponential value when expressing the length as a power of p, p is aninteger such as 2 or 4, it is explained based on the length in thisexample) of the prediction block (M×N), the position of the targetpixel, and the position of the reference pixel. Alternatively, it may beset based on the horizontal and vertical lengths of the prediction blockand the position of the target pixel, and an example described laterwill be developed based on this.

In this case, it is assumed that the position of the pixel is measuredbased on the top left coordinates (0, 0) of the prediction block (M×N).That is, the position of the pixel can be measured by subtracting the xand y components of the top left pixel in the block from thecorresponding pixel.

The reference pixel used in the correction process may belong to a blockadjacent in a horizontal direction (for example, Ref_T, Ref_TR, etc. inFIG. 9), a block adjacent in a vertical direction (for example, Ref_L,Ref_BL, etc. in FIG. 9), and a block adjacent in a horizontal andvertical direction (for example, Ref_TL in FIG. 9) of the current block.

As an example of setting weights, the weight applied to the referencepixel in the adjacent block in the horizontal direction may be set basedon the vertical length (or sum or average of the horizontal and verticallengths) of the current block and the y component of the target pixel,the weight applied to the reference pixel in the adjacent block in thevertical direction may be set based on the horizontal length (or sum oraverage of the horizontal and vertical lengths) of the current block andthe x component of the target pixel, and the weight applied to thereference pixel in the block adjacent to the horizontal and verticaldirections may be set based on the horizontal and vertical lengths ofthe current block (or the sum or average of the horizontal and verticallengths) and the x and y components of the target pixel. In addition,the weight setting may be determined in consideration of the caseopposite to the above example or other variations, and the followingdescribes the case where the weight setting is applied based on theexample.

For example, the prediction mode of the current block is a color copymode, has a size of M×N blocks, and when a pixel located at (i,j) iscorrected, a weight applied to the reference pixel can be set asfollows. In this example, it is assumed that the weights of w_(a) tow_(c) are applied to reference pixels at positions (−1, j), (i, −1),(−1, −1).

S=((Log₂ M+Log₂ N−2)>>2)

w _(a)=32>>((i<<1)>>S)/w _(b)=32>>((j<<1)>>S)

w _(c)=(w _(a)>>4)+(w _(b)>>4)/w _(p)=64−(w _(a) +w _(b) −w _(c))

In the above equation, S is a weight adjustment element obtained basedon the horizontal and vertical lengths of the current block, w_(a) is aweight obtained based on S and the x component of the target pixel,w_(b) is a weight obtained based on S and the y component of the targetpixel, w_(c) is a weight obtained based on S, w_(a), and w_(b). In thecase of w_(c), it may mean a weight obtained based on the horizontal andvertical lengths of the current block and the x and y components of thetarget pixel. Prediction correction values may be obtained by applyingthe weights obtained through the above equation to the correctionprocess equation (however, a shift operation must be performed to theright by 6 in the equation).

In the case of S, it may have a fixed one value, or one of a pluralityof candidates may be adaptively selected. In this case, in the adaptivecase, a plurality of S candidates may be obtained through a method ofapplying a modification (for example, subtract or add 1, 2, etc. to S)of the equation for obtaining the S.

Although the adaptive case for S has been described through the aboveexample, it may be possible that various other weight settings arefixedly or adaptively determined. In the fixed case, it may mean thatone weight setting in an image is used for each block regardless ofencoding/decoding settings, and in the adaptive case, it may mean that aweight setting is determined according to the encoding/decoding settingsand used in a unit of a block. In this case, the encoding/decodingsetting may be defined by one or more of elements such as an image type,color component, block size/shape/position, prediction mode, and thelike.

In addition, for a weight setting, not only one setting fixed to oneblock may be supported, but also two or more settings are supported andadaptively selected in one block as an example of modification may bepossible. As described above, correction may be performed on someregions of a block without performing correction on some regions of theblock. As an extended example, some regions in a block are notcorrected, and some regions (n. N is an integer greater than or equalto 1) in a block may be calibrated according to the weight setting ofeach region.

For example, in the case of a 64×64 block, the bottom right region(Z_(2,2), Z_(3,2), Z_(2,3), Z_(3,3) in FIG. 13 in this example) has along distance from the reference pixel, so correction is not performed.In the top left region (R₀ and C₀ in FIG. 13), since the distance fromthe reference pixel is short, correction can be performed according tothe weight setting A (for example, making the intensity of correctionstronger. use S in the above equation). Since the reference pixeldistance is medium in the middle region (Z_(1,1), Z_(2,1), Z_(3,1),Z_(1,2), Z_(1,3) in FIG. 13) of the two regions, correction may beperformed according to the weight setting B (for example, the correctionintensity is set to the middle. add 1 to S in the above equation).

Alternatively, in the case of a 16×16 block, since the reference pixeldistance is far in the bottom right region (the same as in the exampleabove), correction may not be performed. In the top left region (R₀, R₁,C₀, C₁ in FIG. 13), since a distance from the reference pixel is short,correction may be performed. Alternatively, in the case of a 4×4 block,since all regions have a long distance from a reference pixel,correction may be performed.

The above example shows some cases in which a weight setting in a blockis adaptive according to the size of the block, and examples of variousmodifications are possible without being limited to the above example.In addition, there may be one or more specific boundary value conditionsseparating each region in the above example. In addition, other weightsettings (for example, the number of reference pixels, the position ofthe reference pixels, etc.) may be adaptively set in one block accordingto the position of the pixel to be corrected as well as the S mentionedin the above example.

The prediction mode determination unit performs a process for selectingan optimal mode among a plurality of prediction mode candidate groups.In general, it is possible to determine an optimal mode in terms ofencoding cost by using a rate-distortion technique that considersdistortion of a block {for example, distortion of a current block and areconstructed block. SAD (Sum of Absolute Difference), SSD (Sum ofSquare Difference), etc.} and the amount of bits generated according toa corresponding mode. A prediction block generated based on theprediction mode determined through the above process may be transmittedto the subtraction unit and the addition unit.

In order to determine an optimal prediction mode, all prediction modesexisting in prediction mode candidate groups may be searched, or theoptimal prediction mode may be selected through another decision processfor the purpose of reducing computational quantity/complexity. Forexample, in the first step, some modes having good performance in termsof image quality deterioration are selected for all of the candidates ofan intra prediction mode, and in the second step, an optimal predictionmode may be selected by considering not only image quality deteriorationbut also the amount of bits generated for the mode selected in the firststep. In addition to the above methods, various methods of reducingcomputational quantity/complexity may be applied.

In addition, the prediction mode determining unit may be a configurationgenerally included only in an encoder, but may also be a configurationincluded in a decoder according to encoding/decoding settings. Forexample, in the latter case, where template matching is included as aprediction method or a method of deriving an intra prediction mode in anadjacent region of a current block, a method of implicitly acquiring aprediction mode in a decoder is used.

The prediction mode encoding unit may encode the prediction modeselected through the prediction mode determination unit. In a predictionmode candidate group, index information corresponding to the predictionmode may be encoded, or information on the prediction mode may beencoded by predicting the prediction mode. The former may be a methodapplied to a luminance component, and the latter may be a method appliedto a chrominance component, but is not limited thereto.

When a prediction mode is predicted and encoded, a prediction value (orprediction information) of the prediction mode may be referred to asMost Probable Mode (MPM). The MPM may be configured in one predictionmode or may be configured in a plurality of prediction modes, and thenumber of MPMs (k. k is an integer greater than or equal to 1, such as2, 3, 6) may be determined according to the number of prediction modecandidate groups. When the MPM is configured with a plurality ofprediction modes, it may be referred to as an MPM candidate group.

MPM is a concept supported to efficiently encode a prediction mode. Infact, a candidate group may be configured as a prediction mode having ahigh probability of being generated as a prediction mode of a currentblock.

For example, a MPM candidate group may be configured with a presetprediction mode (or statistically frequent prediction modes. DC, Planar,Vertical, Horizontal, and some diagonal modes, etc.), a prediction modeof adjacent blocks (Left, top, top left, top right, bottom left blocks,etc.), and the like. In this case, the prediction mode of adjacentblocks may be obtained from L0 to L3 (left block), T0 to T3 (top block),TL (top left block), R0 to R3 (top right block), and B0 to B3 (bottomleft block) in FIG. 9.

If an MPM candidate group can be configured from two or more sub-blockpositions (for example, L0, L2, etc.) in an adjacent block (for example,left block), a prediction mode of a corresponding block may beconfigured in a candidate group according to a predefined priority (forexample, L0-L1-L2, etc.). Alternatively, if a MPM candidate group cannotbe configured from two or more sub-block positions, a prediction mode ofa sub-block corresponding to a predefined position (for example, L0,etc.) may be configured in a candidate group. Specifically, among theadjacent blocks, a prediction mode of the L3, T3, TL, R0, and B0positions may be selected as a prediction mode of an adjacent block andincluded in a MPM candidate group. The above description is a case inwhich a prediction mode of an adjacent block is configured in acandidate group, but is not limited thereto. In the example describedbelow, it is assumed that a prediction mode of a predefined position isconfigured in a candidate group.

In addition, when one or more prediction modes are configured as an MPMcandidate group, a mode derived from one or more included predictionmodes may also be additionally configured as an MPM candidate group.Specifically, when the k mode (directional mode) is included in the MPMcandidate group, a mode derivable from the mode (a mode having aninterval of +a, −b based on k. a and b are integers equal to or greaterthan 1 such as 1, 2, 3) may be further included in the MPM candidategroup.

Priority for configuring a MPM candidate group may exist, and a MPMcandidate group may be configured in the order of a prediction mode of aneighboring block—a preset prediction mode—a derived prediction mode,and the like. The process of constructing the MPM candidate group can becompleted by filling the maximum number of MPM candidates according tothe priority. In the case of matching with the previously includedprediction mode in the above process, the prediction mode may not beconfigured in the candidate group, and may include a redundancy checkprocess in which the order is passed to the next priority candidate.

The following assumes that a MPM candidate group is composed of sixprediction modes.

For example, a candidate group may be configured in the order ofL-T-TL-TR-BL-Planar-DC-vertical-horizontal-diagonal mode. It may be acase that a prediction mode of an adjacent block is configuredpreferentially in a candidate group, and a predetermined prediction modeis additionally configured.

Alternatively, a candidate group may be configured in the order ofL-T-Planar-DC-<L+1>-<L−1>-<T+1>-<T−1>-vertical-horizontal-diagonal mode.It may be a case that some prediction modes of adjacent blocks and somepredetermined prediction mode are configured preferentially, and a modederived from the assumption that a prediction mode in a directionsimilar to a prediction mode of an adjacent block will occur and some ofthe preset prediction modes are configured additionally.

The above examples are some cases regarding the configuration of the MPMcandidate group, it may not be limited thereto, and examples of variousmodifications may be possible.

A MPM candidate group may use binarization such as unary binarizationand truncated rice binarization based on an index in the candidategroup. That is, a mode bit can be expressed by allocating a short bit toa candidate having a small index and a long bit to a candidate having alarge index.

Modes not included in a MPM candidate group may be classified as anon-MPM candidate group. In addition, the non-MPM candidate group may beclassified into two or more candidate groups according toencoding/decoding settings.

The following is a premise that 67 modes including a directional modeand a non-directional mode exist in a prediction mode candidate group,and 6 MPM candidates are supported and 61 prediction modes areconfigured in a non-MPM candidate group.

When a non-MPM candidate group is configured as one, since a predictionmode that is not included in the MPM candidate group configurationprocess remains, an additional candidate group configuration process isnot required. Therefore, based on the index in the non-MPM candidategroup, binarization such as fixed length binarization and truncatedunary binarization can be used.

Assuming that a non-MPM candidate group is composed of two or morecandidate groups, in this example, the non-MPM candidate group isclassified into non-MPM_A (hereinafter, A candidate group) and non-MPM_B(hereinafter, B candidate group). It is assumed that A candidate group(p. equal to or more than the number of MPM candidate groups)constitutes a candidate group with a prediction mode that is more likelyto occur as a prediction mode of the current block than a candidate Bgroup (q. equal to or more than the number of the A candidate group). Inthis case, the process of configuring the candidate A group may beadded.

For example, some prediction modes (for example, mode 2, 4, 6, etc.)having equal intervals among directional modes, or a predeterminedprediction mode (For example, a mode derived from a prediction modeincluded in a MPM candidate group, etc.), may be configured in the Acandidate group. The remaining prediction modes through the MPMcandidate group configuration and the A candidate group configurationmay be composed of a B candidate group, and an additional candidategroup configuration process is not required. Binarization such as fixedlength binarization and truncated unary binarization may be used basedon an index in the A and B candidate groups.

The above examples are some cases in which two or more non-MPM candidategroups are configured, it may not be limited thereto, and examples ofvarious modifications may be possible.

The following shows a process for predicting and encoding a predictionmode.

Information (mpm_flag) on whether a prediction mode of a current blockmatches a MPM (or some modes in a MPM candidate group) may be checked.

When it matches the MPM, MPM index information (mpm_idx) may beadditionally confirmed according to the configuration (1, or 2 or more)of the MPM. Thereafter, an encoding process of a current block iscompleted.

When it does not match MPM, if a non-MPM candidate group is configuredas one, non-MPM index information (remaining_idx) can be checked.Thereafter, the encoding process of the current block is completed.

If non-MPM candidate groups are configured in plural (two in thisexample), information (non_mpm_flag) on whether a prediction mode of acurrent block matches some prediction modes in an A candidate group canbe checked.

If it matches the A candidate group, the A candidate group indexinformation (non_mpm_A_idx) may be checked, and if it does not match theA candidate group, the B candidate group index information(remaining_idx) may be checked.

When the configuration of the prediction mode candidate group is fixed,a prediction mode supported by a current block, a prediction modesupported by an adjacent block, and a preset prediction mode may use thesame prediction number index.

Meanwhile, when a configuration of a prediction mode candidate group isadaptive, a prediction mode supported by a current block, a predictionmode supported by an adjacent block, and a preset prediction mode mayuse the same prediction number index or different prediction numberindexes. Refer to FIG. 4 for the following description.

In the prediction mode encoding process, a process of unifying (oradjusting) a prediction mode candidate group for configuring the MPMcandidate group may be performed. For example, a prediction mode of acurrent block may be one of −5 to 61 mode prediction mode candidategroups, and an adjacent block prediction mode may be one of 2 to 66 modeprediction mode candidate groups. In this case, some the predictionmodes of the adjacent block (mode 66) may be a mode that is notsupported for the prediction mode of the current block, so a process ofunifying it in the prediction mode encoding process may be performed.That is, it may be a process that is not required when a configurationof a fixed intra prediction mode candidate group is supported, and maybe a process required when a configuration of an adaptive intraprediction mode candidate group is supported, and detailed descriptionthereof will be omitted.

Unlike the method using the MPM, encoding may be performed by assigningan index to a prediction mode belonging to a prediction mode candidategroup.

For example, if an index is assigned to a prediction mode according to apredefined priority and a prediction mode of a current block isselected, a method of encoding the index corresponds to this. This meansthat a prediction mode candidate group is fixedly configured and a fixedindex is allocated to the prediction mode.

Alternatively, when the prediction mode candidate group is configuredadaptively, the fixed index allocation method may not be suitable. Tothis end, an index is allocated to a prediction mode according to anadaptive priority, and when a prediction mode of a current block isselected, a method of encoding the corresponding index can be applied.This makes it possible to efficiently encode the prediction mode bychanging the index assigned to the prediction mode due to the adaptiveconfiguration of the prediction mode candidate group. That is, theadaptive priority may be to allocate a candidate that is likely to beselected as a prediction mode of a current block to an index in which ashort mode bit occurs.

In the following, it is assumed that 8 prediction modes including apreset prediction mode (directional mode and non-directional mode) inthe prediction mode candidate group, a color copy mode, and a color modeare supported (chrominance component).

For example, it is assumed that four preset prediction modes aresupported among Planar, DC, horizontal, vertical, and diagonal modes(Diagonal down left in this example), and one color mode (C) and threecolor copy modes (CP1, CP2, CP3) are supported. The basic order ofindexes allocated to a prediction mode may be given as a presetprediction mode—a color copy mode—a color mode, and the like.

In this case, the directional mode, the non-directional mode, and thecolor copy mode, which are preset prediction modes, can be easilyclassified into prediction modes in which prediction methods aredistinguished. However, in the case of the color mode, it may be adirectional mode or a non-directional mode, and there may be apossibility of overlapping with a preset prediction mode. For example,when the color mode is a vertical mode, a case may occur in which thevertical mode, which is one of the preset prediction modes, overlaps.

In the case of adaptively adjusting the number of prediction modecandidate groups according to encoding/decoding settings, the number ofcandidate groups may be adjusted (8->7) when the overlapping caseoccurs. Alternatively, if the number of prediction mode candidate groupsis kept fixed, when the overlapping case occurs, an index may beallocated by adding and considering other candidates, which will bedescribed later on the assumption of this setting. In addition, anadaptive prediction mode candidate group may be a supportedconfiguration even when a variable mode such as a color mode isincluded. Therefore, when adaptive index allocation is performed, it canbe regarded as an example of the configuration of the adaptiveprediction mode candidate group.

The following describes a case where adaptive index allocation isperformed according to a color mode. It is assumed that a basic index isallocated in the order ofPlanar(0)-Vertical(1)-Horizontal(2)-DC(3)-CP1(4)-CP2(5)-CP3(6)-C(7). Inaddition, it is assumed that index allocation is performed in the aboveorder when the color mode does not match a preset prediction mode.

For example, when a color mode matches one of preset prediction modes(Planar, Vertical, Horizontal, DC mode), the prediction mode matchingthe index 7 of the color mode is filled. The index of the matchingprediction mode (one of 0 to 3) is filled with a preset prediction mode(Diagonal down left). Specifically, when a color mode is the horizontalmode, index allocation such as Planar(0)-Vertical(1)-Diagonal downleft(2)-DC(3)-CP1(4)-CP2(5)-CP3(6)-Horizontal(7) may be performed.

Alternatively, when a color mode matches one of preset prediction modes,the prediction mode matching the index 0 is filled. Then, the presetprediction mode (Diagonal down left) is filled in the index 7 of thecolor mode. In this case, if the filled prediction mode is not anexisting index 0 (that is, not a planar mode), the existing indexconfiguration may be adjusted. Specifically, when a color mode is DCmode, index allocation such asDC(0)-Planar(1)-Vertical(2)-Horizontal(3)-CP1(4)-CP2(5)-CP3(6)-Diagonaldown left(7) may be performed.

The above examples are some cases of adaptive index allocation, and arenot limited thereto, and examples of various modifications may bepossible. In addition, based on an index in a candidate group, fixedlength binarization, unary binarization, truncated unary binarization,and truncated Rice binarization may be used.

In the following, another example of performing encoding by assigning anindex to a prediction mode belonging to a prediction mode candidategroup will be described.

For example, the method of classifying into a plurality of predictionmode candidate groups by dividing by a prediction mode, a predictionmethod, etc., and encoding by assigning an index to a prediction modebelonging to the candidate group for encoding corresponds thereto. Inthis case, candidate group selection information encoding may precedeindex encoding. As an example, a directional mode, a non-directionalmode, and a color mode, which are prediction modes for performingprediction in the same color space, may belong to one candidate group(hereinafter, S candidate group), and a color copy mode, which is aprediction mode for performing prediction in another color space, maybelong to one candidate group (hereinafter, D candidate group).

The following assumes that nine prediction modes including a presetprediction mode, a color copy mode, and a color mode are supported in aprediction mode candidate group (chrominance component).

For example, it is assumed that four preset prediction modes aresupported among Planar, DC, horizontal, vertical, and diagonal modes,and one color mode (C) and four color copy modes (CP1, CP2, CP3, CP4)are supported. The S candidate group may have five candidates composedof a preset prediction mode and a color mode, and the D candidate groupmay have four candidates composed of a color copy mode.

The S candidate group is an example of an adaptively configuredprediction mode candidate group, and an example of adaptive indexallocation has been described above, so a detailed description thereofwill be omitted. Since the D candidate group is an example of a fixedlyconfigured prediction mode candidate group, a fixed index allocationmethod can be used. For example, index allocation such asCP1(0)-CP2(1)-CP3(2)-CP4(3) may be performed.

Based on the index in the candidate group, binarization such as fixedlength binarization, unary binarization, truncated unary binarization,and truncated Rice binarization can be used. In addition, examples ofvarious modifications may be possible without being limited to the aboveexamples.

Prediction related information generated through the prediction modeencoding unit may be transmitted to the encoding unit and included in abitstream.

In the image decoding method according to an embodiment of the presentinvention, intra prediction may be configured as follows. The intraprediction of the prediction unit may include a prediction mode decodingstep, a reference pixel construction step, and a prediction blockgeneration step. In addition, the image decoding apparatus may beconfigured to include a prediction mode decoding unit, a reference pixelconfiguration unit, and a prediction block generation unit thatimplement a prediction mode decoding step, a reference pixelconfiguration step, and a prediction block generation step. Some of theabove-described processes may be omitted or other processes may beadded, and it may be changed in an order other than the order describedabove.

Since the reference pixel construction unit and the prediction blockgeneration unit of the image decoding apparatus perform the same role asthe corresponding configurations of the image encoding apparatus,detailed descriptions are omitted. The prediction mode decoding unit maybe performed using the method used by the prediction mode encoding unitin reverse.

The methods according to the present invention may be implemented in theform of program instructions that can be executed through variouscomputer means and recorded in a computer readable medium. Computerreadable medium may include program instructions, data files, datastructures, or the like alone or in combination. The programinstructions recorded on the computer-readable medium may be speciallydesigned and configured for the present invention or may be known andusable by those skilled in computer software.

Examples of computer readable medium may include hardware devicesspecifically configured to store and execute program instructions, suchas ROM, RAM, flash memory, and the like. Examples of programinstructions may include machine language codes such as those producedby a compiler, as well as high-level language codes that can be executedby a computer using an interpreter and the like. The above-describedhardware device may be configured to operate with at least one softwaremodule to perform the operation of the present invention, and viceversa.

In addition, the above-described method or apparatus may be implementedby combining all or part of its configuration or function, or may beimplemented separately.

Although described above with reference to preferred embodiments of thepresent invention, those skilled in the art will appreciate that variousmodifications and changes can be made to the present invention withoutdeparting from the spirit and scope of the invention as set forth in theclaims below.

The present invention can be used in an image encoding/decoding methodand apparatus.

What is claimed is:
 1. An image decoding method performed by an imagedecoding apparatus, the method comprising: a block partitioning step forpartitioning a maximum coding unit included in a current image into atleast one coding unit; and a step for decoding the at least one codingunit, wherein the block partitioning step performs at least one among afirst tree-based partitioning and a second tree-based partitioning,wherein the block partitioning step is performed recursively for acoding unit generated by the block partitioning step, and wherein theblock partitioning step for a coding unit generated by the secondtree-based partitioning does not perform the first tree-basedpartitioning.
 2. The method of claim 1, wherein the block partitioningstep is performed based on at least one among first block partitioninginformation on the first tree-based partitioning or second blockpartitioning information on the second tree-based partitioning obtainedfrom a bitstream, wherein the first block partitioning informationcomprises information on whether to perform the first tree-basedpartitioning, and wherein the second block partitioning information isobtained from the bitstream in case the first block partitioninginformation indicates the first tree-based partitioning is notperformed.
 3. The method of claim 2, wherein the second blockpartitioning information comprises partitioning direction information.4. The method of claim 1, wherein a type of tree-based partitioningapplicable for the block partitioning step is adaptively determinedbased on information signaled at a higher level of the current image oran image type of the current image.
 5. The method of claim 4, whereinthe higher level is a sequence level.
 6. The method of claim 4, whereinthe image type of the current image is one among an I-type, P-type orB-type.
 7. An image encoding method performed by an image encodingapparatus, the method comprising: a block partitioning step forpartitioning a maximum coding unit included in a current image into atleast one coding unit; and a step for encoding the at least one codingunit, wherein the block partitioning step performs at least one among afirst tree-based partitioning and a second tree-based partitioning,wherein the block partitioning step is performed recursively for acoding unit generated by the block partitioning step, and wherein theblock partitioning step for a coding unit generated by the secondtree-based partitioning does not perform the first tree-basedpartitioning.
 8. A computer readable recording medium storing abitstream that is generated by an image encoding method, the imageencoding method comprising: a block partitioning step for partitioning amaximum coding unit included in a current image into at least one codingunit; and a step for encoding the at least one coding unit, wherein theblock partitioning step performs at least one among a first tree-basedpartitioning and a second tree-based partitioning, wherein the blockpartitioning step is performed recursively for a coding unit generatedby the block partitioning step, and wherein the block partitioning stepfor a coding unit generated by the second tree-based partitioning doesnot perform the first tree-based partitioning.